Probably the world's leading independent expert

on workplace collaboration & technology 🇺🇦

The Case for Proceeding Without a Data Scientist

Why it may be wise not to wait for one.
Dec | 8 | 2014

Dec | 8 | 2014


It’s hard to find a trendier job these days than data scientist. Big Data has arrived and many organizations are finally starting to explore what it all means. Where are the opportunities? How can they make better, data-based decisions?

It doesn’t take long for many CXOs to realize, however, that their current employees simply aren’t equipped with the skills necessary to find the signal in the noise. This begs the question: What to do?

Culture and Big Data

Before answering that question, let’s take a step back. Lost in much of the hubbub over Big Data is the critical role of culture. As I’ve written before, far too many companies think of Big Data as IT projects or organizational initiatives. This is problematic on two fronts. For one, the last fifteen years have proven that, by and large, organizations fail horribly when deploying new systems and technologies. Beyond that grim reality, projects (IT or otherwise) imply start and end dates. By way of contrast, Big Data is never finished. (Amazon, Netflix, Facebook, and Google are cases in point.) There is more data today than yesterday, both in terms of sheer quantity and number of potentially valuable sources.

Big Data is never finished.

Brass tacks: To make sense out of Big Data, there’s no easy solution or five-point checklist to follow. In fact, organizations might benefit from abandoning—or at least suspending—the search for omniscient data wizards altogether. In “Stop Searching for That Elusive Data Scientist”, Michael Schrage writes about organizations that have attempted jump into the data pool without a proper guide. One example is particularly instructive:

One team, for example, did something as simple as comparing a certain class of tweets from their best customers with their competitor’s customer’s tweets. The overlaps and differences immediately suggested ways to better target and take-away rivals’ customers beyond social media.

Was this team able find every needle in the haystack? Of course not. It would have benefited significantly from the invaluable skills of proper data scientists. Even if this team did not find its Holy Grail, however, exercises like these by “data laypeople” can be tremendously valuable. Team members begin to understand what they can—and, just as important, cannot do—with their organizations’ existing tools. Can it answer sophisticated questions with Microsoft Excel alone? Maybe, but probably not. Perhaps these folks garner a deeper understanding of the issues and available resources.

Put differently, imagine how much more prepared this organization is when the vaunted data scientist actually shows up?

Simon Says

I’m all for data exploration. After all, how do you know what’s out there if you don’t play around and ask fundamental questions? Schrage’s example is particularly instructive. Getting started with Big Data need not immediately necessitate hiring a data scientist, consultant, or other expert.


What say you?

IBM sponsored this post. The opinions in it are mine alone.

Want these posts to arrive in your inbox?

Receive my musings, news, and rants in your inbox as soon as they publish.


Blog E Data E Big Data E The Case for Proceeding Without a Data Scientist

Related Posts

1 Comment

  1. Joshua Ebner

    I definitely agree that they shouldn’t wait, but with a rephrase: they shouldn’t wait for the super-elite machine learning specialists who perform the most arcane forms of data analysis.

    This is like a baseball team waiting for a great pitcher to win baseball games (instead of building a great team of hitters and fielders).

    At the same time, to your point, using Excel probably won’t be sufficient.

    Given the demand, and the supply shortages noted by McKinsey and others, I think the solution is for companies to start re-skilling the Excel-based analysts with better tools.

    Really: the vast majority of analytics work, on the ground, is data wrangling and data visualization. Moreover, visualization is high ROI: it’s much easier to learn (than machine learning); and (echoing your writing) visualization can provide deep insight into business problems.

    If Excel-based analysts “skilled up” over a few months and learned some R, SAS, SQL, and Tableau, they would be vastly more productive. Any solid excel analyst could learn a couple of these tools without too much struggle.

    So, I agree: waiting for the most elite data science specialist is entirely unnecessary. Companies could start to grow great data science teams internally by giving their current analysts the best tools.


Submit a Comment

Your email address will not be published. Required fields are marked *