Award-winning author, dynamic keynote speaker, trusted advisor, & workplace tech expert 


The Folly of the Perfect Prediction

Never conflate the terms better and perfect.
Aug | 4 | 2014


Aug | 4 | 2014

“Perfect is the enemy of good.”

PredictionsWhy does Netflix spend so much money on trying to predict what its 50 million subscribers will watch? (The company famously offered a prize to anyone who could improve its recommendation engine.) Why has it devoted so many resources to collaborative filtering?

How Netflix generates recommendations is a bit of a black box, but the reason is fairly straight forward: Better predictions mean better business. Let me explain.

Think about the two-year contract that you likely have with AT&T, Verizon, or another wireless carrier. If you don’t like the service, there’s not much that you can do in the immediate future. Breaking your existing contract will be expensive, although T-Mobile will cover the fee. Still, for many folks, the devil you know is better than the devil you don’t.

Better does not mean perfect.

Contrast the lock-in business model with that of Netflix. Contract? What contract? You continue paying for as long as you like. If you want to take a few months off, go right ahead sans penalty. Netflix will be there waiting if and when you decide to return. And this is no accident. It’s unlikely that Netflix would have grown as much as fast had it try to lock its customers into multi-year agreements. Reed Hastings and the top brass at Netflix clearly believe that the carrot is a more effective means of garnering customers than the stick. By serving up relevant movies and TV shows, Netflix is betting that its customers will stick around.

Put differently, Netflix is trying to be your friend–or at least not the scourges that most people consider their cable and cell providers to be. Do you think that it’s a coincidence that Netflix blasted Verizon and Comcast on net neutrality?

All of this underscores the importance of timely, relevant, and better predictions. Note that better does not mean perfect. Sometimes Netflix recommends content to me that I find lacking or just plain anodyne. When I stop watching that show after five minutes (and never return), Netflix knows not to recommend similar movies to me in the future.

Simon Says: Keep Expectations in Check

Far too many CXOs justifiably doubt the import of Big Data. Sure, there’s tremendous hype and comparatively little in the way of documented case studies and examples. Ultimately, many ask the bottom-line question: How will this impact my business?

I’ve said many times that it’s critical to manage expectations. Let’s say that you believe that analyzing unstructured data will double your market share or eliminate customer attrition in six months. Good luck with that. Instead, why not take a more measured, conservative approach? Realize that it’s a marathon, not a sprint. If Netflix is still noodling with its recommendation algorithm more than 15 years after its founding, what makes you think that you’ll be able to nail yours in a few months or quarters?

IBM sponsored this post. The opinions in it are mine alone.

Go Deeper


Thoughts on parallels between emerging technologies from last decade and the WFH debate.

Receive my musings, news, and rants in your inbox as soon as they publish.


 Blog E Data E Big Data E The Folly of the Perfect Prediction


Comments close 180 days after post publishes.


Blog E Data E Big Data E The Folly of the Perfect Prediction

Next & Previous Posts