The Data Wars Revisited

Companies are increasingly using data as swords and shields.

Revisited

Nearly eight years ago, Josh McHugh in a great Wired piece asked the question, Should Web Giants Let Startups Use the Information They Have About You? The article examines the pros and cons of allowing small companies to scrape data.

In the nearly eight years since the publication of that piece, scraping data remains a controversial practice. To be sure, there’s significant demand for tools that pull data from websites and return it in usable formats. Startups such as GrepsrKrakiopromptcloud, and import.io allow technical and, in many cases, non-technical users to grab data en masse from websites and create customized application program interfaces (APIs). Put differently, these go well beyond old-school copying and pasting. (Read my interview with import.io CEO David White here.)

import.io screen shot. Click to embiggen.

Across the aisle, many companies view scraping “their” data as a tremendous threat. For instance, the practice represents one way to get yourself banned from Facebook. Zuck understandably doesn’t want people gobbling up reams of Facebook data, without question one of his company’s most valuable assets.

The Larger Trend

As companies grow, they start to restrict access to their APIs.

I’m not going to argue the merits and demerits of scraping here. I do, however, want to call attention to the larger trend going on here. The data wars are not confined to popular sites such as Facebook and Google. The battle for data is becoming increasingly bloody. What’s more, it’s manifesting itself in decidedly unsexy areas such as HR software. (See my post earlier this month on the Zenefits-ADP scuffle.)

  • Build a custom or proprietary API. No longer is the sole purview of tech behemoths.
  • Build a data moat, something that Netflix, Amazon, and Facebook have effectively done.
  • Close or limit access to its API. Many have done this, including Twitter and LinkedIn. Yes, developers can violate the terms of an API and get slapped for doing so.

Simon Says: The data wars have arrived.

To be sure, there are pros and cons with all strategies. For instance, option three might “protect” data, but it’s going to earn the ire of developers and users. Wooing developers and partners by opening “platforms” and APIs is standard practice at the beginning. As companies grow, however, they start to restrict access to their APIs.

In The Age of the Platform, there are no simple answers.

Feedback

What say you?


This post was brought to you by IBM for MSPs. The opinions expressed here are my own.

Navigation

Filed Under



Enjoy this post? Click here to subscribe to this RSS feed or here to sign up for my bi-monthly newsletter.


2 hopefully intriguing thoughts:

  1. Veronica Pullen

    Hey Phil, I was just alerted to your post in my daily alert from Mention, and I wanted to drop by and say thank you for linking back to my post on getting banned from Facebook.

    Since I wrote that post, Facebook have now removed the ability to scrape data completely, by withdrawing access to their api from the scraping software I mentioned. If you try and target an ad to a previously scraped audience, your ad will be denied immediately too,

    Great post, and thank you again for quoting my post.

    Warm regards
    Veronica

    Reply
    • Phil Simon

      You know more about it than I do, but that doesn’t surprise me. You can’t even copy something and pasted for the most part.

      Reply

Submit a Comment

Your email address will not be published. Required fields are marked *