Feeds:
Posts
Comments

Archive for the ‘Technical info’ Category


Many companies have found success in building data warehouses that meet basic needs, but are now finding they need to move beyond the back-office warehouse to leverage information on the front lines of decision making throughout their entire company. They need information on demand and need the ability to build systems that can deliver on those promises with real incremental returns.

For those who understand the power of an analytics-driven organization, this is a most exciting time. The opportunities are limitless: customers, prospects, suppliers and the business itself are creating endless geysers of data. Analytics tools are inexpensive, widely available and so easy to use that they make business sense in almost any situation.

To move forward, organizations need a strategy that delivers on several focused business requirements:
1) Operational management: Accelerate time-to-market to meet business SLAs for new and existing business processes, operational analytics and business intelligence (BI).
2) Big data: Leverage unstructured data, social media and other “big data” information sources to gain more insights from more data—without impacting the business SLAs.
3) Predictive analytics: Forecast future trends and analyze risks and potential outcomes.

Many IT organizations are adopting a strategy called smart consolidation that reconciles the need to simultaneously distribute data warehousing and analytics capabilities and infrastructure while centralizing management. Smart consolidation is a method for evolving an existing data warehouse architecture to meet today’s demanding analytic needs, such as big data, streaming data and unstructured data.

In a nutshell, it involves thinking beyond the traditional warehouse structures that have provided great success with structured data, basic reporting and analysis. Smart consolidation is driven by these four goals:

  1. Consolidate and govern enterprise data
  2. Optimize workloads for performance and SLAs
  3. Simplify the delivery of analytics by leveraging appliances
  4. Flexibly extend analytic capability as needed

The basis for smart consolidation is to completely optimize an analytics architecture by placing the right workload against the right data, in the right place, at the right cost and the right performance level.

Smart consolidation acknowledges that an organization requires different types of databases, analysis tools and data formats. It needs traditional data warehouses, data warehouse appliances and operational BI systems that can accommodate different types of workloads. It also needs systems based on advanced technologies that can efficiently handle data that is moving extremely quickly as well as large volumes of data that does not change frequently.

Single system? I think not

No single, data system could efficiently serve all these requirements and perform well for both transactional and analytical workloads. Under the smart consolidation strategy, multiple specialized elements use industry standards to communicate and join together to form a fluid, agile data ecosystem that delivers business insight, cross-organizational data governance and centralized IT resource management. By allowing many different elements to serve specialized needs, smart consolidation also enables organizations to accommodate the endless variety and rapidly growing ocean of semi-structured and unstructured data.

Read Full Post »


Find out more: http://www-01.ibm.com/software/data/infosphere/warehouse/
Find out more: http://www-01.ibm.com/software/data/db2-warehouse-10/

Scenario: A retail store is now able to predict and replenish its merchandize stock by operationalizing an automated process flow. The retailer can analyze inventory data and product stock levels in real time. By deploying IBM InfoSphere Warehouse to reduce decision latency, the store’s accelerated decision-making capability now effortlessly keeps pace with the speed of demand.

Key benefits of InfoSphere Warehouse v10:
Faster Insights

  • Real-time Business Intelligence as operational data is continuously being feed into the warehouse.
  • Faster, accurate decision making, turnaround times.

Improved Cost Efficiencies

  • Adaptive Data Compression provides on average 30% improvement (up to 75%) over IBM’s existing Deep Compression.
  • Multi-Temperature Storage allows you to optimize data storage cost-efficiency.

Higher Performance

  • Star Schema optimization delivery for quicker response times – delivering 3x performance on BI workloads.
  • Continuous Ingest of data optimizes loading of data leading to faster decision making.
  • High Availability Operational access concurrent with analytics.

Increased Team Productivity

  • Built-In Time Travel query enabling faster historical and trend analytical queries.
  • Row and Column access controls to support multiple tenant operational warehouses.
  • Native Bi-Temporal support improves developer and DBA productivity.

Read Full Post »


Find out more: http://www-01.ibm.com/software/data/infosphere/warehouse/

Facing the limitations that can hamper access to insight

Many existing data warehousing solutions have limitations, such as a lack of real-time operational data, that make it difficult for organizations to fully leverage the power of information. 

  • Limited reach: Business users need to access and analyze a broad array of information, from the unstructured information in call center notes, emails and blogs to the structured information in databases, spreadsheets and other data sources. Yet current infrastructures often have limited reach. The proliferation of transaction systems and the emergence of new data sources available outside the business force users to cast a wide net to gather the information they need to support better decision making.
  • Limited access: Enterprises must deliver useful and timely information to more people as part of everyday business processes to drive innovations and reduce costs. Business intelligence (BI) and analytics tools can help, but only if they are widely available rather than limited to high-level decision makers and specialized analysts.
  • Limited depth: Many existing information infrastructures also have limited depth. Often business users are unable to get answers to complex ad hoc questions. IT teams need flexible tools that can capture and deliver more types of information in the way that users need it, when they need it, where they need it and how they need it.
  • Poor flexibility: Older BI systems may restrict the type and quantity of data that certain users can access. Organizations need ways to support a large numbers of diverse users and enable those users to quickly and easily customize how they receive information based on their specific needs.
  • Poor responsiveness: In many cases, existing information systems lack the responsiveness required by today’s business landscape. Organizations can gain a significant advantage over competitors if they can quickly analyze a wide range of information and deliver actionable insights to executives and frontline decision makers. They need solutions that can be implemented swiftly and then deliver rapid results.
  • Excessive costs and complexity: Some BI solutions can be costly to acquire, difficult to deploy and complex to manage. Organizations need solutions that can be implemented quickly and include capabilities that streamline administration and lower the overall cost of ownership.
  • Lack of real-time operational data: Without the availability of accurate, real-time operational data, business decision making can stall and organizations may miss identifying opportunities and insights. Several factors contribute to a lack of real-time operational data. For example, archaic extract, transform and load (ETL) processes can lead to delays in capturing operational data into the data warehouse. Performance overheads in handling heavy workloads and complex queries from the data warehouse for analytics processing can also cause problems. Within the data warehouse, non-responsive resource allocation according to the priority and criticality of data can slow analytic results.

Organizations must address these challenges and work to achieve on-demand access to insight. If they succeed, they can target smaller customer segments and communicate with them about their individual needs and wants. They can identify and capitalize on even the smallest trends, attaining competitive advantages normally only realized by more flexible and dynamic businesses. They can detect small behavior patterns that can have a significant influence and impact on business in terms of revenue, expenses and growth. Most important, they can build competitive strategies around data-driven insights and generate impressive business results.

Gaining insight without boundaries using IBM InfoSphere Warehouse

Through dynamic warehousing, IBM helps organizations extract insight from virtually any type of data—helping to deliver the right information at the right time and in the right context so business leaders can make the right decisions quickly. IBM dynamic warehousing solutions integrate data warehousing and business analytics to help define an organization’s central business concepts and the data required to support those concepts. These solutions allow organizations to pull data from a variety of enterprise and source systems that traditional BI and data warehousing solutions have not been able to access in the past. As a result, IT organizations can better support business requirements for actionable information. This information is not just raw data but data backed by intelligence that can help people take action and make sound business decisions.

IBM InfoSphere Warehouse is a complete, multipurpose environment that allows organizations to access, analyze and act on operational and historical information, whether structured or unstructured. With InfoSphere Warehouse, organizations can gain the insight and agility they need to generate new opportunities, contain costs and satisfy customers. Unlike traditional data warehouse and BI solutions, which may be complex and inflexible, InfoSphere Warehouse simplifies the processes of selecting, deploying and maintaining an information management infrastructure while offering the flexibility for dynamically integrating and transforming data into actionable business insights. It enables organizations to centrally, accurately and securely analyze and deliver information as part of their operational and strategic business applications.

Read Full Post »


Find out more: http://www.ibm.com/software/data/infosphere/warehouse/
Virtual Conference: https://events.unisfair.com/index.jsp?seid=33250&eid=556 

IBM InfoSphere Warehouse is a complete real-time data warehousing platform that delivers superior scalability and availability, design, build, and management tooling, and business analytics. It provides a complete, multipurpose environment that allows organizations to access, analyze and act on operational and historical information, whether structured or unstructured.

With InfoSphere Warehouse, organizations can gain the insight and agility they need to generate new opportunities, contain costs and satisfy customers. Unlike traditional data warehouse and BI solutions, which may be complex and inflexible, InfoSphere Warehouse simplifies the processes of selecting, deploying and maintaining an information management infrastructure while offering the flexibility for dynamically integrating and transforming data into actionable business insights. It enables organizations to centrally, accurately and securely analyze and deliver information as part of their operational and strategic business applications.

New and enhanced features of version 10:

One of the driving motivations behind this new version 10 is the desire to accelerate time-to-insight so that decision makers throughout an organization can make faster (and better) business decisions. Accelerating time-to-insight focuses on two key areas: 1) the latency of data ingest, and 2) the analytics latency – time to perform analytics and provide insights to decision makers.

Business value is realized when a decision is made or action taken.

Reducing Latency improves business value by being able to make decisions faster.

Continuous ingest

In more traditional warehouse environments, additional data is loaded once a day, once a week or perhaps even once a month. InfoSphere Warehouse continuously ingests data from multiple sources throughout the organization while optimizing resource utilization across all the data sources.

Continuous ingest of data optimizes loading of data – not once per day, not once per hour, but continuously. This ensures that the most up-to-date data is used all the time – no more waiting for a nightly ETL process and scheduled reports to be run the next morning. This is a significant feature for companies where being first is critical – ha, I can’t think of any company that wouldn’t want to analyze fresh data and take action before their competitors!

Multi-temperature data management

As warehouses grow in size, storage costs escalate. At the same time, service-level agreements (SLAs) for warehouse access are becoming tighter for certain mission-critical warehouse information. Multi-temperature data management enables warehouse data to be spread across multiple tiers of storage, defined by performance or cost. The optimizer and workload management features use information about data priority to enhance query performance for hot data while reducing resources for lower-priority queries that access archival or cold data.

Faster performance delivers insights faster. Pretty simple concept. Multi-storage data management puts high priority, frequently accessed data in the fastest (hot) storage available to the warehouse, and of course, less frequently accessed data in warm and cold storage. This accomplishes two things – 1) for end users, queries run faster, and 2) for IT, it optimizes performance, storage and costs.

Storage savings and enhanced performance through adaptive compression

Adaptive compression improves on the leading compression technology found in prior versions of InfoSphere Warehouse by adaptively determining the optimal compression between row- and page-based compression. Using InfoSphere Warehouse 10.1, organizations can improve the compression ratio by nearly 30 percent compared with prior versions. By combining InfoSphere Warehouse and the terabyte charge metric (in which organizations pay for only the post-compression data volume), organizations now can store more raw warehouse data with less storage while enjoying improved performance to query that data. Adaptive compression eliminates the need to completely rebuild compression dictionaries as data grows and reduces the requirement to periodically perform table reorganizations.

Time travel queries through Temporal Table support

The relevance of data within a warehouse often degrades significantly over time. For example, knowing the specific updates to a customer’s record throughout the day may provide valuable insight into a customer’s activity patterns. Two days later, however, it may no longer be necessary to store same-day activities data within the warehouse. Temporal Table support enables organizations to enhance table definitions to define the relevance of transitional changes within the rows stored in the table. After changes reach a predefined threshold, historical (temporal) changes are removed from the table. Organizations can use temporal analytics to gain new insights into the patterns and transitory activities of their business—insights that were previously too expensive to manage or store within the warehouse.

Improved efficiency and performance for BI queries

To improve the speed of BI queries, InfoSphere Warehouse provides industry-leading efficiency in collating multiple data formats, including unstructured data. The result is a 3x or better performance improvement for standard, star-schema types of complex BI queries. Intelligent analytic query optimization helps reduce I/O to minimize resource requirements for heavy workloads.

Advanced security through row and column access control

Advanced access control allows organizations to restrict data access to certain users or groups. Administrators can decide to present only the data that users are allowed to see, or they can mask certain data, such as all but the last four digits of credit card numbers. Quick setup and standardization of security features simplifies security, enabling organizations to control, monitor and help ensure enforcement of security by the database. These security policies help reduce complexity for organizations trying to manage these policies and procedures within applications and business groups. Fine-grain access control capabilities also enable organizations to host multitenant warehouses while limiting access to a single tenant’s information, thereby leveraging resources across multiple warehouse clients.

Real-time operational warehousing

Using the features and enhancements of InfoSphere Warehouse 10, organizations can realize the benefits of real-time data warehousing and deliver greater business value through faster access to time-critical data. Historical trend analysis using temporal tables allows finer insight into business and market trends. By maximizing the utilization and prioritization of resources, organizations can achieve those benefits at a lower total cost of ownership.

Read Full Post »


 

The IBM Netezza data warehouse appliance

IBM Netezza data warehouse appliances combine advanced hardware, software and storage inside a true appliance. The result – a purpose-built, highly optimized, high performance environment for analytics.

Need answers to your most pressing business questions? Drop in your data, lots of it, and press the EASY button to gain new insights into your data. Read the article here…

Read Full Post »


Good data is attained by integrating multiple data sources, deriving a ‘single version of the truth,’ and putting that good data (and unstructured content) into a data warehouse where the BA/BI tools can perform their magic. DDD (data-driven decision making) begins and ends with good data.

“smarter technology” posted a great article “Data Two, Gut One” that got me thinking more about the value of good data. In the article, they state that new research shows data-driven decisions improve organizational performance and company value.

In my current position as a marketing evangelist for data warehousing and analytics, that has proven itself time and time again. When looking for ways to increase our marketing effectiveness, I look at the various marketing channels we use to get our messages out to the market. Having complete data available regarding marketing activities, marketing channels used, customer segmentation sets, response rates…and, how leads progress through the sales funnel, is critical.

When some of the data is unavailable, quite simply, the “truth” is not known, and any decisions based on this information is just a guess, gut feel, intuition, a hope and a prayer, a SWAG. When the data is complete, accurate and trusted, I can then make quality decisions to fill in any gaps, go after new markets, tweak the messaging to get higher response rates, etc.

Analytics applications that nicely present dashboards, scorecards, historical trends, predictive analys, and give me actionable insights, can all benefit from good data. Good data begins with data integration, data quality, and a good data warehouse.

If anyone reading this post has had good experience with good data, or a bad experience with bad data, I encourage you to share your story by commenting.

Read Full Post »

%d bloggers like this: