The search is on for the new search

Google are shaking in their boots. They’ve indexed nearly every single website that ever existed but their algorithm works on the principles of yesteryear. Boiled to the basics, Google rank a site based on the number of, relevance of, and quality of links pointing to it. The difficulty comes with “no follow” links, which are basically links that Google choose to ignore. With the invention of web2.0 having no follow links made sense, with sites open to mass participation they were also open to mass spamming. Having “no follow” links stopped spammers increasing their search ranking by spamming sites with links.

A storm of problems has been heading Google’s way since January, when Twitter exploded into the mainstream and attracted users from all over the world. It’s now super easy to share links on the microblogging service, and that’s exactly what people are doing. The collective link generation of our mass mind is creating quite a database of information which twitter can search in real time. The problem is turning a search of peoples tweets into a useful service?

twitters-growth1

image coutesy of mashable http://mashable.com/2009/05/07/facebook-twitter-myspace-growth-april/

Tweetmeme now offer realtime twitter search capabilities, each and every time you post a link to twitter, tweetmeme spiders crawl and index it. This opens up the door for a new type of search engine, one that ranks websites based on how many people talk about them rather than how many sites link to them. People are the future, not websites. Twitter need to hurry up and develop this system themselves before their colossal database is indexed and sold on to users by third party websites.

The trend in “real time search” leads us to OneRiot, a search engine that indexes the links Google has been ignoring for years – links shared by normal people on social networks. OneRiot indexes the links on twitter, reddit, digg, and so on, making their search capabilities very much in the “what’s hot right now” territory. It’s very quickly turning into the social search engine and could pave the way for a new kind of ranking system that’s much harder to game. It’s been easy enough to game Google’s algorithm to increase the rankings of a particular site, article marketing, link baiting, hidden links, and other SEO techniques have meant hitting the top spot for a search term is not about providing the best service, it’s about spending the most money on SEO. A search engine that ranks a site based on what people are saying about it in real time is going to produce more accurate results and maybe we’ll see the rise of “social optimisers” who work relentlessly to improve the way a website or service is talked about in the social arena.

If you’ve been reading the blog regularly you’ll know my thoughts on google reader’s aesthetic; it’s butt ugly. The new search engines springing up are far from ugly, they’re fun and inviting to use, and their eye candy is making Google look naked and shamed. Some time and attention has actually been taken to make them look like the future while Google’s software increasingly looks like yesterdays.

A good example of a nice aesthetic is a new search engine called Kosmix, we’re pleased with out initial experience of it. It produces good search results and organises them into relevant boxes instead of just producing a long ugly list of links for you to scroll through. Take a look at how nice their explainer video is!

The tides of search are most certainly turning but it’s not all roses for the new search engines. They’re still unlikely to topple the Google megalith any time soon simply because of the monstrous size of Google’s indexed database. AllGoogle need to do is slightly change their algorithm, stop ignoring a few “no follow” links and add some sort of “what’s going on right now” feature.

Well, you may or may not be pleased to hear that this is exactly what they’ve done. Some new features rolled out yesterday allow searchers to filter out results based on the time published, their type (forum post, blog post, video, review etc) and they even have a new spangly “wonder wheel” in some vein attempt at looking cool. A quick test of the features shows that they’re already indexing twitter updates and if you’d like us to speculate they’re probably crawling and indexing the links twitter too. The holy grail of search will use an algorithm that ranks a site based on peoples discussion of it, rather than the links generated to it. Twitter half hint towards this idea with their search “smiley faces or sad faces” filter but if anyone is capable of really pulling it off it’s probably Google, maybe they’ve got this search game on lock down.

They still look like s**t though.

Phil Harper - Social Media Consultant