Over the last few weeks, I’ve read plenty of articles that refer to Big Data as projects. Here’s one.

This notion of Big Data as an IT project is a real bone of contention for me. It represents in my view one of the most pernicious myths out there on the topic. Big Data is fundamentally different than implementing ERP and CRM systems. Beyond that, I don’t understand why were so eager to classify Big Data as projects anyway. As I wrote in my first book, IT projects don’t exactly have stunning success rates, and that’s with tried-and-true methodologies and mature systems.

Are We Finished Yet? Not Google

You won’t find this project mentality at Big Data behemoths like Google. Case in point: the company just retooled its search engine–again. As CBS News reported:

The overhaul came as part of an update called “Hummingbird” that Google Inc. has gradually rolled out in the past month without disclosing the modifications.

The changes could have a major impact on traffic to websites. Hummingbird represents the most dramatic alteration to its search engine since it revised the way it indexes websites three years ago as part of a redesign called “Caffeine,” according to Amit Singhal, a company vice president. He estimates that the redesign affects about 90 percent of the search requests Google gets.

The obvious question is, Why? Why do this when you still control more than two-thirds of the US search market? After all, in many old-school companies, even half of that market share would lead to a culture of complacency. You’d hear many arguments like, “It ain’t broke, don’t fix it.”

Keep refining algorithms. Keep using data

Two main reasons come to mind. First, Larry and Sergey understand all too well The Innovator’s Dilemma (affiliate link), the classic business text by Clayton Christensen. Companies like RIM/BlackBerry, Kodak, and Microsoft are among the scores that have stagnated or fallen from grace because they failed to embrace new ways of doing things. In each case, the world changed–and not just in a minor way. Tectonic changes were taking place. Each organization minimized the long-term impact of new threats and technologies. Self-preservation trumped innovation. Second, and more germane to this post, Google understands the dynamic nature of data. The sources of data change, as do their uses. Couple that an improved understanding about how the Web works, and you get Hummingbird.

Simon Says: Big Data Is Never Finished

Any “project” that results in a 67 percent share of the US search market must be considered successful, but that term implies definitive start and stop dates. Google doesn’t make this mistake. I’d bet a great deal of money that Hummingbird will not represent the last incarnation of Google’s search engine. Nor should it.

Now, I have no inside knowledge of the ins and outs behind Hummingbird, and odds are that you don’t either. In a way, though, those particulars are irrelevant. The larger lesson is that most companies ought be acting like Google does. Keep evolving. Keep refining algorithms. Keep using data. Learn from one of the most formidable companies on the planet. Data evolves. So should any products or services that hinge upon it.

In other words, stasis is the enemy of progress.

Feedback

What say you?

I wrote this post as part of the IBM for Midsize Business program