Tom's Ten Data Tips - January 2011

Self Service BI

Business intelligence (BI) creates value by empowering knowledge workers. Self service BI is a relatively new trend in response to the ever growing need for timely, useful information. It embodies an acknowledgement that meeting these needs should be agile and flexible.

Whenever IT professionals attempt to manage information requirements, “translation errors” and responsiveness stand in the way of immediate fulfillment of requests for actionable data. Self service BI puts end users firmly in the driver’s seat, and much ‘closer’ to their data.

1. Query Access To Source Systems Doesn’t “Cut It”

Some end user requests can be rightfully met by granting “direct” access to source systems. This has some advantages, too. IT (or your BI department) only needs to be minimally involved, data are always current, and insights (or odd query results…) drive awareness about the need to ensure data quality. Data ownership comes more “naturally.”

The drawbacks tend to outweigh these advantages, though. Especially in the long run. Most source systems have third normal form (3NF) data models which are poorly suited for query access. Besides performance issues, they require advanced SQL skills. Kimball: “A freshman in business needs a PhD in SQL.” So-called “embedded analytics” deals with this modeling problem, but does not necessarily facilitate self service BI. To make data actionable usually requires some level of data integration (combining multiple data sources), and that is where BI has a chance to shine.

2. Self Service BI Shifts Focus From Data Management To Information Gathering

In “traditional” BI teams, a lot of focus goes to enabling business users to do their job by providing accurate and timely access to data. These are still needed and essential for self service BI. However, instead of managing data, self service BI is all about enabling business users to do their jobs better. The nature of cooperation between BI teams and business end users helps BI professionals get a much more profound insight in how end users are turning data into value for the company. This facilitates alignment between IT and end users so they both focus exclusively on meeting information needs.

3. No BICC Can Constrain Information Needs

Unfortunately, all too often BICC’s act as emperors in their data fiefdom. BICC’s should be appointed the single source of corporate approved data. However, this should not mean they are the sole source of data per se. This data monopoly might be seductive, because it grants tremendous power. Any hint of abuse of this power invariably leads to mushrooming of alternate spreadmarts, and home grown data warehouses (see also tip# 9).

You just can’t stop the incoming tide: needs for data are growing, demands are rising, and at the same time the opportunity and ability to process in some “DIY” fashion have grown dramatically. New tools are making this easier and easier, and a new generation of “data customers” just won’t put up with unresponsive BI professionals. This puts additional strain on your data governance. Simply disqualifying an Excel spreadsheet won’t help. At all.

4. There’s A Reason Why Excel Is (Still) The Most Used BI Tool

Many BI professionals are outright negative about Excel. The most commonly asked question right after finishing a (successful) BI project is “How can I export the results to Excel?” Then end users create little works of art, fragile and brittle as they may be.

Research shows that unaudited Excel sheets have errors in them in 94% of cases, and 5.2% of cells contain errors. This is clearly a source of confusion and misunderstanding, and almost the opposite of the BI holy grail of “one version of the truth.” Then why is Excel (still) so popular? This, in large part has to do with the control it allows end-users. So rather than “fight” this need, isn’t it about time we address those needs with self service BI?

5. Self Service Doesn’t Mean No Service

One of the hugely powerful features of self service BI is that end users can not only create, but also distribute information products (like dimensional cubes). This has turned over unparalleled power to data analysts. Sometimes “spreadmarts” can become a liability when too few people can maintain them, possibly only one individual. What happens is that critical processes become reliant on them, and key staff have turned into a single point of failure. Self service BI holds similar risks.

When you monitor data usage, you’ll be able to assess which information products might be candidates for an “upgrade” to corporate IT products. Professional maintenance may become required as usage spreads, and primary process dependencies emerge. Just make sure this transition “back” into IT is painless and user-friendly, or skunk works stealth applications might just drop off your radar (see also tip# 3).

6. Self Service BI (Still) Needs To Be Managed

The promise of self service BI is that you enable maximum flexibility and autonomy for end-users, yet at the same time can avoid “spreadsheet hell” (you know it, don’t you?). This requires automatic updates with fresh (or corrected) data and also comprehensive monitoring of usage. That’s all easier said than done.

The focus is on the front-end of solutions, and that is fine. However, to make a managed self service BI solution ‘work’ requires clever design and build of your BI (data warehouse) back-end.

7. Data Ownership Promotes Quality

Self service BI gives end users firm ownership of their own information (and data). When inconsistencies arise, they can quickly toggle between source systems and BI data sources. By making this loop as short as possible, and in particular by enabling knowledge workers to provision their own information needs, you get a powerful feedback loop. And nothing will drive your data quality forward as fast as feedback and insight in where gaps arise.

8. Governance And Guiding Principles Create Common Understanding

The key to truly successful self service Bi is to enable end users maximum freedom, yet at the same time tame the ensuing data chaos. Users may have unprecendented power and freedom, as long as this doesn’t lead to anarchy and confusion.

You know you are successful as soon as a shared understanding of your business emerges. To get there, you’ll need solid governance based on educated users and guiding principles on how and when to distribute information products. More databases are only better as long as you avoid an information glut.

9. Data, Data, Everywhere…

In the “old” days, source systems and a central DWH together would comprise 95%+ of all data in use for business intelligence. But increasingly, new sources of data are making their way into the marketplace. ZIP code and life style (or psychographic) data have been around for a long time. Some countries have (public) census data, and often derived data are available, too.

New initiatives like for instance Microsoft’s “Azure” offer a genuine marketplace for data. Tools like Google’s @@@ help end-users become proficient at merging data sources. Unresponsive IT departments will cut themselves off “true” information needs: if end users are willing to go this route, for sure it demonstrates a genuine need. Either they proceed to work with Excel (see also tip# 4), or they can become part of mainstream BI plans. Information needs will get fulfilled one way or the other.

10. Power To The People

Self service BI brings data analysis to knowledge workers and decision makers. In modern day corporations, a power shift has happened. No longer is hierarchical structure the dominant source of power, but instead the people who can turn data into knowledge become sources of (informal) power. This drive has created an information democracy where influence stems from an ability to make sense of vast amounts of data.

But ‘simply’ acquiring PowerPivot (a neat new Microsoft BI product), doesn’t mean your self service BI is good to go. As always, tools are an important, arguably necessary part of the equation. But they’re never a sufficient condition, and that point is easily lost on some sales people. This transformation into information democracy is about taking data governance to the next level, and handing power over from IT to knowledge workers.