The End of Big BI?

Are its days numbered?
Mar | 22 | 2012

Mar | 22 | 2012

In the mid- to late 1990s, many organizations undertook large business intelligence (BI) projects at considerable expense. The vast majority of these companies deployed BI applications via a sequential or Waterfall methodology, often taking a year or more to implement. While the particulars varied from project to project, most were conceived as follows:

  • IT gathers requirements from business users, including the all-important key performance indicators (KPIs).
  • IT designs a data model to support those requirements and KPIs.
  • IT identifies data sources that need to be loaded into a cube or data warehouse.
  • IT loads the data into its target, and formulates a process by which to extract, transform, and load the data (aka, ETL).
  • IT rolls out the final product to end users, often in the form of visually compelling dashboards, cubes of data, alerts, and other ad hoc reporting tools.

In theory, these types of deployments were supposed to meet the needs of different lines of business (LOBs). Sadly, they often did not. Many if not most BI projects did not meet often the lofty expectations of CIOs who believed that these new tools would transform their businesses. While the reasons for these project failures varied, organizations characterized as early adopters often served as examples for others of what not to do. What’s more, they often deterred many mid-market companies from even going down this road. Relatively few organizations could justify expensive software licenses, consulting fees, and hardware upgrades—especially when their own core CRM and ERP applications were lacking.

BI 2.0: More Palatable for the Mid-Market

So, what’s different today for the mid-market organization vis-à-vis BI? Many things, actually, but I’ll focus on two in this post. First, BI no longer needs not be a massive undertaking, both in terms of time and cost. On the time side, in many organizations, Agile software development is replacing the Waterfall method as the primary means by which to deploy applications. Rather than a “big bang” at the end of a year or more of work, users see bits and pieces of the application as it develops. Think iterations. Users can identify problems, issues, or inconsistencies early on, thus dramatically reducing the amount of rework.

The halcyon days of Big BI are coming to an end—and that’s great news for many mid-market organizations.

On the cost side, we’ve seen the maturation of many BI applications and the increasing popularity cloud computing. As such, organizations need not spend $1 million or more on new databases and servers to launch a powerful BI tool. Yes, many BI applications are still memory- and resource-intensive, but lighter and less taxing options abound. Costs also drop because IT’s role has changed: It may no longer need to be the main gatekeeper (and, many times, bottleneck), particularly if an organization opts for BI as a service.

Note that IT departments are still essential to the successful deployment of on-premise BI applications. In a cloud-based environment, however, IT’s role may be smaller, but it’s by no means insignificant. For many technical, logistical, and even regulatory/legal reasons, IT still serves an important function for organizations embracing cloud computing.

Simon Says

The halcyon days of Big BI are coming to an end—and that’s great news for many mid-market organizations.


What say you??

I wrote this post as part of the IBM for Midsize Business program.

Want these posts to arrive in your inbox?

Receive my musings, news, and rants in your inbox as soon as they publish.


Blog E Technology E The End of Big BI?

Related Posts

Excerpt From The Nine

ChatGPT is all the rage, but generative AI is a powerful force. It is so much more than a single tool.

1 Comment

  1. Stray__Cat

    Ditto Brother!
    I have been working along these lines for years and it was rewarding and much more fun!


Submit a Comment

Your email address will not be published. Required fields are marked *