Award-winning author, dynamic keynote speaker, trusted advisor, & workplace tech expert 


Past ≈ Future?

Thoughts on whether Big Data will result in more accurate forecasts.
Jul | 16 | 2014


Jul | 16 | 2014

Most of us have heard this line–or something similar–more than a few times:

The best predictor of future behavior is past behavior.

As a longtime sports fan, I’ve seen this bromide play out more times than I can recall. One of my favorite examples here concerns baseball. Many statistics like batting average, hits, wins, RBI, walks, and the like almost always resemble a bell curve. That is, absent injury, a strike-shortened season, or other extenuating circumstances, the trajectory of a player’s productivity tends to peak mid-career and then wane as Father Time catches up.

That’s not to say, though, that there haven’t been plenty of notable sports exceptions to the “past ≈ future” rule. For instance, Oriole outfielder Brady Anderson’s home run totals had historically been pretty predicable, give or take. Then, from 1995 to 1996, the number skyrocketed from 16 to 50.

Yes, 50.

Due strictly to chance or skill, an increase of more than 200 percent is completely implausible–that is, at least due to natural events. Of course, Anderson’s productivity boon stemmed from artificial means, so much so that he made the All Steroid Team.

And then there’s Apple. The chart below shows when I, on the advice of ostensibly experienced analysts, bought more than a few shares of the company’s stock:

Click on image to embiggen.

Click on image to embiggen.

That hurt.

Clearly there’s no direct relationship between past and future performance when it comes to individual stocks. If it were, then investing would be downright easy.

How will Big Data impact the relationship between past and future performance? Put differently, will we be able to read between the lines better?

It’s a lofty and interesting subject, one that Nate Silver addresses in his excellent book The Signal and the Noise: Why So Many Predictions Fail —but Some Don’t (affiliate link). Perhaps

Simon Says

The era of Big Data and predictive analytics guarantees nothing. I strongly suspect that Black Swans will continue to exist.

There’s been no shortage of exceptions to the “past ≈ future” rule.

I’ll be the first to admit that this is tough for many people to swallow, but there’s a good chance that predictions continue to miss their marks. I for one believe that many, if not most things, will only completely understood in hindsight. Big Data can certainly help, but I sincerely doubt that we’ll get to the point at which our models can take everything into account. I suspect that the future never completely resemble the past.

While the words and opinions in this post are my own, Trendensity has compensated me to write it. 

Go Deeper

Receive my musings, news, and rants in your inbox as soon as they publish.


 Blog E Data E Analytics E Past ≈ Future?


Comments close 180 days after post publishes.


Blog E Data E Analytics E Past ≈ Future?

Next & Previous Posts