Originally published on the Amazon tech store site.
With so much hype these days, it’s not hard to understand why so many people are confused about Big Data. After all, two years ago, virtually no one used the term. Today, nothing could be further from the truth. Just about everyone has something to say about Big Data, adding to the confusion.
While researching Too Big to Ignore: The Business Case for Big Data, I discovered many pervasive myths about Big Data. In no particular order, here are some of the biggest Big Data myths:
- Big Data is only for big companies. Organizations of all sizes–and in all sectors—are currently realizing major benefits from Big Data. In the book, I provide a number of examples and case studies. Companies nowhere close to large (in terms of revenue and employees) are utilizing Big Data solutions as we speak.
- Big Data works with old tools. Relational databases and SQL statements don’t work well with petabytes of unstructured data. New solutions like Hadoop, NoSQL, and columnar databases are necessary in order to make Big Data happen.
- Big Data is a fad. This is perhaps the biggest of Big Data myths. Companies like Amazon, Apple, Facebook, Google, Twitter, and others have shown that Big Data is here for the long term. If you think that Big Data is going away, think again. Mobile devices, broadband connections, the decline in data storage costs, and the arrival of the cloud collectively mean that the data generated and consumed will continue to explode.
- Big Data is predictable. The Gangnam Style video (viewed more than one billion times on YouTube) and the SuperBowl blackout (and millions of associated tweets) prove that there’s no planning for a new trend an explosion of Big Data. It can’t be controlled and erupts very quickly.
- Big Data is internal to the organization. The vast majority of data these days lies outside of the walls of even the largest organizations. Make no mistake: Internal data management is still important. However, the data outside of the enterprise is arguably more essential than external data.
- Big Data is neat, orderly, and structured. Big Data is actually unstructured, messy, and tough to “manage” in the traditional sense.
- Big Data is a luxury. More and more, Big Data is becoming essential for organizations to succeed.
- Big Data is just noise. To any given person, department, or organization, most data is irrelevant. As Facebook shows, however, Big Data allows organizations to match the right person or group to the right event, page, or trend.
- Big Data is static and easy to tame. Big Data is extremely dynamic. New sources like Pinterest can erupt almost overnight and become valuable sources of information.
- Big Data is complete and perfect. Organizations will never be able to capture every conceivable piece of data—and there’s really no need. In the era of Big Data, filters are still critical–and maybe more so than ever.
- Big Data only stems from people. Increasingly, machines are generating much of what we call Big Data. To be sure, the Internet of Things will only intensify this trend.
Depending on whom you ask, Big Data is a flash in the pan, something that will revolutionize business, or something in between those two extremes. While I have no crystal ball, put me squarely in the ‘revolutionize business’ camp. In order for business leaders and organizations to realize any value from the petabytes of unstructured data streaming at us faster than ever, it’s imperative to look beyond Big Data solutions like Hadoop, NoSQL, and columnar databases. First and foremost, we need to develop a common understanding of new terms. Buzzwords, jargon, and myths only serve to confuse people at a time when communication is essential.
To find out more about big Data, check out my latest book.