NEW WORKSHOPS ADDED

Phil Simon

THE WORLD’S LEADING INDEPENDENT WORKPLACE COLLABORATION & TECH EXPERT

The Data Wars Revisited

Companies are increasingly using data as swords and shields.
Jul | 7 | 2015

Jul | 7 | 2015
}

Introduction

RevisitedNearly eight years ago, Josh McHugh in a great Wired piece asked the question, Should Web Giants Let Startups Use the Information They Have About You? The article examines the pros and cons of allowing small companies to scrape data.

In the nearly eight years since the publication of that piece, scraping data remains a controversial practice. To be sure, there’s significant demand for tools that pull data from websites and return it in usable formats. Startups such as GrepsrKrakiopromptcloud, and import.io1 allow non-technical users to grab data en masse from websites and create customized application program interfaces (APIs). Put differently, these go well beyond old-school copying and pasting.

import.io screen shot. Click to embiggen.

For those with mad Python chops, libraries such as Beautiful Soup and Scrapy can typically go well beyond what WISWYG scrapers can do.

Across the aisle, many companies view scraping their data as a tremendous threat. They’re not wrong. For instance, the practice represents one way to get yourself banned from Facebook. Zuck understandably doesn’t want people gobbling up reams of Facebook data, without question one of his company’s most valuable assets.

The Larger Trend

As companies grow, they start to restrict access to their APIs.

I’m not going to argue the merits and demerits of scraping here. I do, however, want to call attention to the larger trend going on here. The data wars are not confined to popular sites such as Facebook and Google. The battle for data is becoming increasingly bloody. What’s more, it’s manifesting itself in decidedly unsexy areas such as HR software. (See my post earlier this month on the Zenefits-ADP scuffle.)

  • Build a custom or proprietary API. No longer is the sole purview of tech behemoths.
  • Build a data moat, something that Netflix, Amazon, and Facebook have effectively done.
  • Close or limit access to its API. Many have done this, including Twitter and LinkedIn. Yes, developers can violate the terms of an API and get slapped for doing so.

Simon Says: The data wars have arrived.

To be sure, there are pros and cons with all strategies. For instance, option three might “protect” data, but it’s going to earn the ire of developers and users. Wooing developers and partners by opening “platforms” and APIs is standard practice at the beginning. As companies grow, however, they start to restrict access to their APIs.

In The Age of the Platform, there are no simple answers.

Feedback

What say you?

Footnotes

  1. Read my interview with import.io CEO David White here.

Receive my musings, news, and rants in your inbox as soon as they publish.

 

Blog E Data E Big Data E The Data Wars Revisited

Related Posts

Outliers

ognitive decline terrifies me because, like many of you, I make my living with my brain. To keep it as spry as possible, I do a number of things. My morning ritual involves drinking coffee and playing several New York Times games. Wordle and...

Thoughts on Twitter and Section 230

Section 230 of the Communication Decency Act is getting plenty of attention these days. My friend Josh Bernoff reached out to me for an op-ed that he's penning for The Boston Globe. I misunderstood that he just wanted a short quote. Instead, I started thinking and...

Don’t Be Evil by Rana Foroohar

Building upon books such as World Without End and Weapons of Math Destruction, Don't Be Evil: How Big Tech Betrayed Its Founding Principles -- and All of Us makes the case that Big Tech is doing more harm than good. Rana Foroohar proves her central thesis in spades. I...

The Wild Wild West of Analytics Programs

s I write these words, I'm in the midst of teaching my fourth year of analytics courses at ASU. To be sure, it feels longer than that. That's probably because, during this time, I have done more than merely fulfill my 4/4 teaching load. I wrote a...

2 Comments

  1. Veronica Pullen

    Hey Phil, I was just alerted to your post in my daily alert from Mention, and I wanted to drop by and say thank you for linking back to my post on getting banned from Facebook.

    Since I wrote that post, Facebook have now removed the ability to scrape data completely, by withdrawing access to their api from the scraping software I mentioned. If you try and target an ad to a previously scraped audience, your ad will be denied immediately too,

    Great post, and thank you again for quoting my post.

    Warm regards
    Veronica

    Reply
    • Phil Simon

      You know more about it than I do, but that doesn’t surprise me. You can’t even copy something and pasted for the most part.

      Reply

Submit a Comment

Your email address will not be published. Required fields are marked *

 

Blog E Data E Big Data E The Data Wars Revisited

Next & Previous Posts

Related Posts

Outliers

ognitive decline terrifies me because, like many of you, I make my living with my brain. To keep it as spry as possible, I do a number of things. My morning ritual involves drinking coffee and playing several New York Times games. Wordle and...

Thoughts on Twitter and Section 230

Section 230 of the Communication Decency Act is getting plenty of attention these days. My friend Josh Bernoff reached out to me for an op-ed that he's penning for The Boston Globe. I misunderstood that he just wanted a short quote. Instead, I started thinking and...

Don’t Be Evil by Rana Foroohar

Building upon books such as World Without End and Weapons of Math Destruction, Don't Be Evil: How Big Tech Betrayed Its Founding Principles -- and All of Us makes the case that Big Tech is doing more harm than good. Rana Foroohar proves her central thesis in spades. I...

The Wild Wild West of Analytics Programs

s I write these words, I'm in the midst of teaching my fourth year of analytics courses at ASU. To be sure, it feels longer than that. That's probably because, during this time, I have done more than merely fulfill my 4/4 teaching load. I wrote a...

2 Comments

  1. Veronica Pullen

    Hey Phil, I was just alerted to your post in my daily alert from Mention, and I wanted to drop by and say thank you for linking back to my post on getting banned from Facebook.

    Since I wrote that post, Facebook have now removed the ability to scrape data completely, by withdrawing access to their api from the scraping software I mentioned. If you try and target an ad to a previously scraped audience, your ad will be denied immediately too,

    Great post, and thank you again for quoting my post.

    Warm regards
    Veronica

    Reply
    • Phil Simon

      You know more about it than I do, but that doesn’t surprise me. You can’t even copy something and pasted for the most part.

      Reply

Submit a Comment

Your email address will not be published. Required fields are marked *