At a
recent webinar, David Hatch from Aberdeen Group showed some survey results about the likely reasons for the inability to delivery timely BI for business decision makers.
The results reinforce intuition: 59% said ability to integrate data from multiple sources.
Distant second and 3rd place were "ease of use for non-technical end users", and "lack of well defined user needs", which were 18 and 9 percent respectively.
My comment: This last one, the 9% on "lack of well-defined user needs", is the real issue here and is responsible for most of both the ease-of-use pain and the multi-source integration pain.
Let me elaborate here a bit. Without a clear business focus on what the BI solution is trying to do, then there is no possibility of creating an easy-to-use product for non-technical folks. Think about it for a minute. Non-technical user... means they really want something tuned to what they understand, but if the user needs aren't well defined then how can something be tuned to things that aren't well defined? Non-technical user ease-of-use and lack of well-defined user needs are contradictions in terms.
In addition, ability to integrate data from multiple sources is also to a great extent about well-defined user needs. Yes, there's the difficulty of just getting the data extracted, but most people can solve that issue. The problem is this: Without a specific business focus area there is no QA context to let you know whether your integration processing is correct or not. Without an understanding of the business model that addresses the user needs there is no ability to integrate data together.
The whole way our industry has focused on BI is a layered approach. The layers at the bottom are source systems, then there is a data warehouse layer, then tools/marts, then business users at the top. The problem is not the pieces in the layers. Rather, it's a naive layer-at-a-time way of thinking about implementing them. Many BI projects seem to think you can build this stack layer by layer from the bottom up. People working on the bottom layers don't have to understand the business problem, that's for the people working near the top.
Question: how do people building the integrated data warehouse check their work? What provides them with evidence that the information is correct and complete for any given purpose? All this data governance and stewardship stuff we constantly hear about in data circles - all stuff that is very hard to realize - is an effort to assure data quality in the absence of a real business QA context for the data.
At Oco we use a completely different approach which addresses these questions head on. It cuts across the layers of the stack and is driven from top to bottom by well defined user needs. Our process begins with a set of predefined guideline reports and we conduct a business-profiling session where customers select and specify variations on these reports to address their business needs focusing on where there is an acute pain. This same community of users (who are non-technical usually) provides the QA context which allows the integration of data, even from multiple source systems, to be done with a limited amount of effort.
Oco uses a proven and patented Model-Driven Data Integration (tm) approach, which accelerates the whole data integration process by relating source transactional data to our versatile, cross-industry BI data model. But the whole reason this mapping process works is again due to the overall focus on solving a particular BI problem. Only data that is needed for the solution is mapped and more importantly, is subject to QA. There's no point warehousing data of dubious quality, and the business solution provides QA context for the data it needs, but not for other data from those same source systems. This makes integration much easier. You don't pay attention to data that isn't part of the solution, not because it wastes space or time, but because you can't QA that data.
In summary, it's as simple as this:
(1) no focused business needs implies no QA context implies you can't do data integration across sources.
(2) no focused business needs implies no ease-of-use for non-technical users.
Focus on the business problem - is what makes the whole BI world easier to grapple with, more effective, and lower cost.