The Real McCrea

Have you ever found yourself refreshing your browser while watching the river of content in your aggregator of choice, wondering why your latest photos, blogpost, or tweets haven’t shown up? “Are they there now?” Pause. “How about now?” Sigh.

A company launching today, Gnip, seeks to cure the “aggregation latency” problem, by flipping the model from one in which the aggregators periodically pull data from all the different social media sites to one in which updates are pushed to them from a single service (Gnip).

Is there really a need for such a service?

When Plaxo launched Pulse last August, kicking off a wave of social media aggregation that would spawn FriendFeed, Iminta, Social Thing, among others, the various social media sites, like Twitter, Flickr, and Digg, were not not expecting them. They had not built out infrastructure to handle a rising tide of requests for updates. Each aggregator had to figure out how frequently it could send out its crawlers without running into throttles at the various social media sites. And each social media site had to figure out how best to handle this new load coming from other services, rather than directly from their users.

All of the parties are now wrestling with scaling this model. And seeing that, the guys at Gnip saw an opportunity. It’s really interesting to watch the Social Web ecosystem beginning to emerge. As my readers know, I’m a big believer that aggregation is a critical core of that ecosystem, because in an era when most of the Web is social, aggregators provide the only way for mainstream users to keep up with what their family and friends are creating and sharing online.

Joseph Smarr of Plaxo has a nice post, explaining it in more detail. Also ReadWriteWeb’s Marshal Kirkpatrick has thoughtful analysis.

[Reminder/disclosure: I work at Plaxo, one of the companies discussed in this post.]