John Grisham. Steven King. Jack Canfield. Each author has sold hundreds of millions of books. Yet, before they were A-list authors, established traditional publishers rejected manuscripts from each of them, forcing them to either self-publish or to work with very small independent presses.

And the converse is true. More than 70 percent of all books don’t “earn out.” That is, authors fail to sell enough books to pay their publishers back their initial advances. The money that the author receives upon signing a contract most often represents the only check they will receive. Ever.

The forces of inertia are remarkably powerful in many industries, and publishing is no exception. As Michael Meyer wrote in 2009 New York Times’ piece:

Yet despite the economic downturn, and the fact that 7 out of 10 titles do not earn back their advance, the system doesn’t seem to be going away anytime soon. In recent interviews, a dozen New York-based publishers and agents told me, more or less, “Publishers have to keep buying books,” and “They have to bid for the best books” — which in large part means those that will sell.

Publishing in 2014

The Grisham and King examples might seem dated to some. Let’s look at something more contemporary. Surely publishers have improved their batting averages in the last five years, right? Perhaps not. Case in point: the ostensibly inexplicable bestselling book by an obscure French economist in 2014. Thomas Piketty’s 700-page tome on income inequality, Capital in the Twenty-First Century, is the very definition of a surprise bestseller, and not just among laypersons. Even smartypants economists were shocked. “When the book came out,” says economics professor Brad DeLong, of the University of California at Berkeley, “I thought it had an audience of three people in Berkeley”—himself, economic historian Barry Eichengreen, and Christina Romer, former head of the president’s Council of Economic Advisers.

Big Data does not mean the end of uncertainty.

You’ll get no argument from me on the state of traditional publishing and its inability to adopt to changing times. That’s one reason that I started my own micropublisher in 2010. Having penned more than a few books, I know where the bodies are buried. As such, let me say this about the book acquistion process: most publishers and acquisitions editors typically use at least rudimentary data when making the “buy/don’t buy” decision. Data include the prospective author’s:

• Previous book sales
• “Platform” (i.e., social media numbers like Twitter followers, Facebook likes, etc.)
• Number of mailing list recipients
• Number of previous and forthcoming speaking gigs
• Proposed topic/competition (e.g., Is it another social media book?)

Although better than merely relying upon gut feeling, using data like this doesn’t come remotely close to successfully predicting future book sales. There’s just no way to account for a bevy of unknown factors, not the least of which is luck.

Simon Says

We can certainly get better at making predictions. Big Data offers tremendous promise. Still, it’s important to manage expectations here. People who believe that new software, methodologies, and consultants will eliminate uncertainty are bound to be disappointed. There’s a world of difference between managing the future and predicting it.

This post is brought to you by SAS.