relevance, your honour?

The search engine business is a tough one. People are generally pretty bad at knowing how to phrase queries to give them what they want, to say nothing of dealing with spelling mistakes and synonyms and stemming, and you have to do all that work basically instantaneously. The relevance of search results might be the only thing more important than performance in determining if users will stick with your particular product, or make the trivial switch to another one.

So I was pretty surprised to discover how, er, idiosyncratic the search results were on Live Search for what I — perhaps naively — think of as a pretty straightforward query.

When searching for “Firefox”, the user might want to find the home page for the product, or a description of the history of the project, or maybe even a review of the software. Both Yahoo and Google give you some mix of that, with what seem to me to be pretty reasonable orderings of results.

The Live Search results are a little more difficult for me to understand, since they have the Silverlight developer FAQ as the first result, then an article about cross-site scripting, then an article about ASP.NET, and then the Wikipedia page about Firefox. You have to go to the 8th entry to get the product’s home page, well below the fold on my machine at least. I’ve saved off the results, in case you disbelieve me, or for some reason can’t reproduce them yourself.

Maybe Live Search users really are a different breed, if that’s what they would be most likely to want when searching for Firefox; a ballsy market-differentiation move by Microsoft, if so.

(Canadians don’t call their judges “Your Honour”, and Americans don’t spell honour that way, so the title of this post is a somewhat impossible reference, but I figure you’ll let that slide.)