The Case for Small Data Analytics

Why the emphasis on the Big stuff is often misplaced.
Feb | 2 | 2015

Feb | 2 | 2015

Amidst all of the hype and jargon surrounding Big Data, perhaps no term is as hackneyed as Big Data analytics. Ask ten thought leaders, consultants, or salespeople what this means and you’re bound to receive ten different answers.  

Here’s a definition as good as any:

Big Data analytics enables organizations to analyze a mix of structured, semi-structured, and unstructured data in search of valuable business information and insights.

No doubt that, equipped with the right tools (e.g., Hadoop) and mindset, organizations can unearth fascinating insights into their troves of data. In some cases, they’ll be able to this independently. In others, though, they might need to enlist the help of partners.

I often wonder, though, if the sexy nature of the term Big Data inadvertently inhibits many individuals, groups, departments, and companies from effectively using their Small Data. You know, the structured, relational database-friendly stuff.

An Example

In December, I experienced a number of über-frustrating broadband outages. The problems required me to call my provider 25 times, including 16 during one particularly irritating three-day period. (Calm blue oceans, I kept telling myself.) To be sure, the content of each call constituted the very definition of unstructured data.

The provider’s recording claimed that “all calls may be recorded to ensure quality assurance.” We have all heard that message before. In theory, there would be a call detail record (CDR) for each of my exchanges at minimum. The larger question is, What do employees actually do with recordings that don’t go viral? (See Ryan Block’s Comcast call as an example of a company forced to deal with a crisis of its own doing.)

Not nearly enough, as far as I’m concerned.

Organizations would be wise to make better use of their Small Data.

Forget the substance of each conversation for a moment (read: the data); just the sheer number of my calls over such a short period of time should have served red flag to anyone in the company’s customer retention department. Yet, I heard nothing crickets. Putting aside my frustrations as a customer for a moment, the data guy in me was amazed at what was happening—and not happening with the data and metadata I generated over the course of that month. Given the situation, I couldn’t shake the following queries:

  • Why wasn’t anyone looking at my tweets and phone calls and reaching out to me?
  • Why was the onus always on customers to report their issues and status updates?
  • Why wasn’t anyone calling me to ensure that my service appointment actually took place?
  • Why didn’t anyone call me after I had stopped calling? Or didn’t anyone even notice?

Rest assured that if I ran this company, things would be dramatically different. Forget some vague notion of “empowering” a retention department and encouraging “engagement.” Give me data, damn it. Entry-level phone reps would immediately know when customers called and which were at-risk of leaving. Dashboards would augment—if not altogether replace—individual e-mails. The right people would receive alerts. I would hold employees accountable for responding to them beyond perfunctory Twitter responses of “I’m sorry you’re having issues. Please give us your address.” After all, how hard can it be to store a customer’s Twitter handle in her profile?

Simon Says

There’s no question that organizations can learn a great deal from truly unwieldily amounts of data. I have zero doubt, however, that those same organizations would be wise to make better use of their Small Data.


What say you?

IBM sponsored this post. The opinions in it are mine alone.

Receive my musings, news, and rants in your inbox as soon as they publish.


Blog E Data E Analytics E The Case for Small Data Analytics

Related Posts

How to Switch LC/NC Vendors

When it comes to low-code/no-code, I know a thing or six. I couldn't have written the new book otherwise. Here are two of them. First, vendor consolidation is an inevitability. There's just no way that the status quo is sustainable. Consider the vast amount of choice...

Book Review: Wonder Boy

In 2011, I moved from NJ to Vegas. It didn't take long for me to hook up with the Vegas tech scene and the Downtown Project. Over the course of my five years in Sin City, I attended events, spoke at Zappos's HQ a few times, met plenty of smart cookies, and learned a...

Publication of HBR Book on Agile Methods

A few months ago, I announced that Harvard Business Review was including one of my contributions in its forthcoming book Agile: The Insights You Need from Harvard Business Review (HBR Insights Series). I'm pleased to announce that that book is now out and available...

1 Comment

  1. Grayson

    You’re spot on with companies getting distracted with sexy “big data.” I’ve yet to see “customer frustration” as a metric that can be adequately extracted and acted upon by number crunching. Put down the spread sheet and have a conversation with your customers every once in a while…


Submit a Comment

Your email address will not be published. Required fields are marked *