Chlorine for the “Cesspool:” Why Google, Microsoft, and Yahoo Will Battle to Win Twitter

by Steve Broback on April 7, 2009

After several months of pontificating offline about this issue, I have finally decided to commit it to keystrokes. In a nutshell, it lays the case for why we built a conference about Twitter, and why we feel the service is a key strategic acquisition target. There is much more to the story than what I present here, but this captures the essence. If you want to offer the best Web search engine, you need to buy Twitter.


The bottom line is that whoever acquires Twitter will in essence take possession of an army of millions (soon to be tens of millions) of humans who are actively, accurately, and enthusiastically meta-tagging pages. In the arena of human-augmented search, Mahalo is a useful wheelbarrow, while Twitter is a fleet of 747 cargo planes. The search engine that integrates Twitter data properly will likely become recognized as the “best” search engine out there.

Let’s consider the search landscape and the vast fortune in stock value that Google has acquired (significantly at Yahoo’s expense.)

In the mid-1990’s Yahoo was the prime default destination site for Web surfers. It was the starting point for most looking to find something on the Web. It used a simple, but effective method of determining what site was relevant to what search terms — it analyzed what words were on the page, and heavily weighted the meta-tag information that the author entered to categorize the content. The closer this tagged text aligned with a search term, the higher it placed in search results.

The problem was that this was quickly gamed by those who are trying to sell things online. Soon content and tags no longer could be used as the central measurement of relevance. One would search for “best digital camera” and find pages dedicated to Viagra. This would often be a page whose tags and invisible background copy would say one thing, but the big image file on the center of the page would say another.

No worries, Google introduced a scheme where inbound links — “citations” would determine where a page placed. This would work great for quite some time. No reasonable human is inclined to link to giant banner ad for Viagra, so search results tended to be clean and correct. People migrated in droves to this new and better provider.

The problem is that Google is now being effectively gamed. Have you performed a google search lately? Does anyone remember the pristine results we saw in back in 1997? If you do, you likely agree with Google CEO that the Web has become a “cesspool” populated with splogs and irrelevant marketing drivel — and Google isn’t filtering it out particularly well. It’s not their fault, it’s just that the spammers are innovating big-time.

Consider what happens if Twitter data is incorporated into Google, Yahoo, or LiveSearch. Any tweet that contains a link effectively serves as a citation as well as a meta-tag for a page. This tag can be weighted for relevance by any number of factors (location, time, retweets, followership, etc.) I’m not saying it can’t be gamed, but I am saying it’s difficult to do.

Many bloggers are already talking about how is the now the first place they now go to get the information they want. Many others have read this writing on the wall as well.

While it’s true that the data can be integrated now, and a potential acquirer could conceivably just pay Twitter for the information, but in that scenario they’d lack the proprietary edge that is desperately needed.

{ 15 comments… read them below or add one }

DL Byron April 24, 2009 at 9:07 am

One of your best ever titles. I don’t even need to read the rest of the post, that was so satisfying.

Mike Koss May 26, 2009 at 7:38 am

I find that Twitter searches return as much if not more spam as Google. Twitter has done notthing to rank content for quality or relevence as most people use search to find the most recent tweeets for their search term.

And its arguably a harder problem to identify the chaf as there is much less data about a single tweet to base a decision on.

Philippe Dame May 26, 2009 at 7:40 am

Twitter can and is already “gamed”. It will get worse when it really hits mainstream. I’m already swamped with Twitter spammers that follow me just so I can view their profile one time (i.e. “Get Viagra now” + a URL is their last status update).

Twitter has poor SPAM control and if Google can’t detect and remove it all, how will Twitter? That said, I agree it still would be a smart and strategic acquisition for a search engine to add that human-filtered and live element.

SPAM, however, will spoil it for many of us. It’s tough for a few companies to out-innovate legions of profit-seeking hackers.

Adina Levin May 26, 2009 at 7:43 am

Twitter as a means of adding metadata, absolutely.

But Twitter isn’t a cure for spam. The amount of Twitter spam is increasing. Spam is endemic to any successful interactive service, unfortunately. Twitter’s owners will need to fight spam forever to keep it for killing the service.

jay May 26, 2009 at 7:43 am

makes complete sense, if you’re searching for promotional material, ironic videos, celebrity gossip, and other blog posts. try searching for actual, substantive content on and you’ll get a whole lot of useless, unsourced results.

Dan Parsons May 26, 2009 at 10:31 am

The spam issue on twitter is getting very serious indeed. I believe the first step toward a solution will be a reputation score based system similar to that in place on One’s ability to follow masses of people on twitter needs to be in proportion to their reputation score, and there needs to be some non-trivial algorithm based on legitimate forms of participation over time.

Shane McCallum May 26, 2009 at 10:59 am

I couldn’t agree more with Philippe, Twitter is chock full of spam and “gaming”. I have used StackOverflow and I like Dan’s idea, though the algorithm would have to be something truly special for it to work on such a massive, and diverse, populace of users.

Thanks to Steve for his article, since before this I wasn’t really sure I understood why the big 3 were so interested in Twitter. BTW I found this post via a tweet. :)

Mark Essel May 27, 2009 at 3:26 am

Couldn’t agree more with you Mr. Broback. I believe many heavy users of web, search and social media have seen powerful advantages of human scoured and filtered pages (digg, stumbleupon, reddit, etc) with commentary/metatags.
This is a wonderful opportunity for a smaller search engine to gain massive strides on Google. Friendfeed search is pretty fantastic, and they do much more than search.

Phil Baumann May 27, 2009 at 8:51 am

Found this post on FriendFeed.

As it is now, Search.Twitter isn’t nearly as powerful as it could be. Will Twitter evolve their own TweetRank algorithm, I don’t know. Perhaps it will take an acquisition for that to happen.

It’s still not clear who will win the status race: FriendFeed isn’t very mainstream but it does appear to capture metadata well and its search is more pliant than Twitter’s right now.

The comments above about spam are important for Twitter to take note. Twitter would certainly be a more attractive acquisition if it dealt more effectively with spam. Spam will always be a never-ending problem and I suspect that the so-called real-time web will be a never-ending stream of challenges and opportunities.

Chlorine for the cesspool: yes, please, can we have some already?

Kir August 11, 2010 at 3:14 am
Jane August 25, 2010 at 12:42 am
Diesel August 27, 2010 at 2:13 am
Aron August 27, 2010 at 2:19 am
louis vuitton neverfull January 18, 2012 at 4:55 am

Personally, I find that to generally be one of the fascinating topics in the event it draws a parallel to.

accident attorney littlerock ca February 7, 2012 at 9:01 am

Leave a Comment