relevance, your honour?
The search engine business is a tough one. People are generally pretty bad at knowing how to phrase queries to give them what they want, to say nothing of dealing with spelling mistakes and synonyms and stemming, and you have to do all that work basically instantaneously. The relevance of search results might be the only thing more important than performance in determining if users will stick with your particular product, or make the trivial switch to another one.
So I was pretty surprised to discover how, er, idiosyncratic the search results were on Live Search for what I — perhaps naively — think of as a pretty straightforward query.
When searching for “Firefox”, the user might want to find the home page for the product, or a description of the history of the project, or maybe even a review of the software. Both Yahoo and Google give you some mix of that, with what seem to me to be pretty reasonable orderings of results.
The Live Search results are a little more difficult for me to understand, since they have the Silverlight developer FAQ as the first result, then an article about cross-site scripting, then an article about ASP.NET, and then the Wikipedia page about Firefox. You have to go to the 8th entry to get the product’s home page, well below the fold on my machine at least. I’ve saved off the results, in case you disbelieve me, or for some reason can’t reproduce them yourself.
Maybe Live Search users really are a different breed, if that’s what they would be most likely to want when searching for Firefox; a ballsy market-differentiation move by Microsoft, if so.
(Canadians don’t call their judges “Your Honour”, and Americans don’t spell honour that way, so the title of this post is a somewhat impossible reference, but I figure you’ll let that slide.)
Those are some interesting ads at the top. Firefox 2008? Sweet.
be afraid of monopolies?…
Mike Shaver has an interesting post up about search results. I’m going to assume that this is a case of incompetence rather than abusive and anti-competitive behavior — not that they deserve that assumption of innocence…….
It seems their results are very heavily location dependent, with searches for ‘google’, ‘yahoo’, and ‘microsoft’ all leading me to UK regional sites for those companies (despite me being in Ireland). Maybe the bad Firefox results are a symptom of that?
Incidentally, circuit judges in the UK and Ireland are referred to as ‘Your Honour’, so your title reference isn’t broken.
Interestingly, if you increase the “popularity” slider (Advanced > Results ranking) for the “Firefox” search – even just one point – the results are ordered much more how you expected.
I’m also mildly interested as to whether this “fixes” it for everyone else (search for “Firefox {popl=51}” if you don’t want to play with the UI).
Being in london, having de_DE as my first country code I get http://www.mozilla-europe.org/de/ http://www.firefox-browser.de/ http://www.firebird-browser.de http://www.netzwelt.de/browser/firefox.htm (Titled: Firefox, the better browser) http://www.firefox2.de Pretty foxy! So it does depend on your settings.
@Rory: Have you got enUK or enIE in your settings? Tools->Options->Advanced->Languages in FF.
In California I’m seeing pretty similar results to what Shaver sees up in the soon-to-be-frozen northland (with a 1:1 exchange rate), although the ads are a little different (I see one for Firefox-2007 instead of Firefox-2008, and the order of the last 2 is switched), and the top 5 search results are the same except for the third (I get a rather different MSDN page). The product page is 8th; mozilla.com is 9th, but the mozilla.org project page is 6th (and a mozilla.org keybord shortcut help page 7th).
I get better results from NZ too.
I hate to be paranoid, but Microsoft makes it so easy!
Shows better results (but still not ideal) here in Australia.
Mozilla Firefox http://www.mozilla.com/firefox
Firefox 5 Minute Challenge lachy.id.au/dev/mozilla/firefox/ campaign/5minute/challenge
Mozilla CEO speaks out on future of Firefox | APC Magazine apcmag.com/6054/firefoxceo speaks_out
Firefox – Opera – Andrew Gregory’s Web Pages http://www.scss.com.au/family/andrew/opera/ firefox
Mozilla Firefox Web Browser – Firefox Download for Free firefox.org.nz
Firefox News firefox.org/news
Firefox Help: Firefox FAQ http://www.mozilla.org/support/firefox/ faq
Mozilla Firefox – Wikipedia, the free encyclopedia en.wikipedia.org/wiki/Mozilla_Firefox
Silverlight FAQ is first here in Prague as well. Microsoft isn’t doing itself any favors with this type of thing considering that it already has a tough road to hoe to compete with Google in search, without biasing its engine towards its own products (as this result strongly implies).
Not that Google doesn’t display some idiosyncrasies of its own. For example, “shaver blog” took me to a journal about removing facial hair instead of to this page, as it obviously should.
It seems that the search results are dependent on browser locale – despite being in Germany, I see the same results as Shaver in my en-US Firefox. But I must say, the sorting of search results is generally pretty crappy in Live Search, so I might agree with Asa that this is simply a case of extreme incompetence.
I tried it on the Russian version (live.ru) and it returns the download page first. Odd.
Five years ago, I would have read this and immediately assumed that the cause was ‘evil’. Now I’m pretty split between ‘evil’ and ‘incompetent’ as potential causes…
The results vary if depending on your browser. What this article reports is what IE shows. If you run the same query on Fx or Opera you get the “expected” results…
Very interesting
What do you mean, Canadians don’t call their judges Your Honour?
First results page from Turkey:
http://www.mozilla.com/en-US/firefox http://www.mozilla-europe.org/tr/products/ firefox http://www.firefox.fr http://www.mozilla.org/projects/firefox firefox.hu http://www.mozilla-japan.org/products/firefox http://www.firefox-browser.de firefox.no
Posting from Costa Rica, I see the same results as shaver, with Silverlight leading the result list. Ugh.
Those are some pretty horrible results from Live.com. Always be specific when you search. Search for [firefox web browser] and you’ll get better results. The Opera web browser, suprisingly, shows up as the first result regardless of whether you search for [opera] or [opera web browser]. It’s strange that Opera, that shares its name with a genre of music, is better understood by the Live.com search engine than Firefox, which is fortunate enough to have a completely unique name.
I envy the name Firefox sometimes because it’s cool sounding and unique, so people’s first impression isn’t a fat lady screeching in Italian.
I get the same results as shaver and Wladimir here in Germany, but I also have an en-US browser locale.
@shaver, btw, what do the Canadians call their judges? Germans say “high court”, at least that’s what I heard them say at the supreme court.
In the law classes I took, we were told that it was either “my Lord/Lady” or “Mr./Madam Justice”, though it seems that the former is now considered inappropriate:
http://cas-ncr-nter03.cas-satj.gc.ca/portal/page/portal/fccfen/FAQ#20 for example.
Here from Malaysia with EN locale, the result for Firefox is still Firefox website – http://www.mozilla.com/firefox/
The Firefox 2008 ad is unnerving though. Mozilla advertised in AdCenter?
Interestingly, now I get exactly the same search results as before – but the first two hits (Microsoft) are no longer there. The Firefox download page is still rather far down.
The Live Search results are a little more difficult for me to understand, since they have.. . No man, you know the answer: “Our products on top, then everything else”. And about MSN search in general I agre with http://mynull.blogspot.com author MSN, Google, Yahoo provide the same results.
I took a quick look at ranking for a mozilla.com or mozilla.org domain for organic search on MSN Live, Google, and Yahoo for the following search terms:
“firefox” “Mozilla” “Mozilla Firefox” “firefox download” “firefox.com” “download firefox”
Basically, all of these terms were #1 in all three search engines for all but one term — “firefox” — which was #4 in MSN live search (seems there is some variation from other folks posting here, but it’s definitely not #1). And for some reason, the listing that came up in MSN at #4 was “www.mozilla.org/projects/firefox” — not crazy though, considering that the url has “firefox” in it, “Firefox” is mentioned in the title tag and throughout the copy of the page, etc.
It seems to me that MSN Live search algorithms aren’t totally dialed-in yet (or maybe just not that great to begin with) as the other two big kids on the block.
The new version of Live Search does much better. My first result – from Redmond :) – is http://www.mozilla.org/projects/firefox/.
In Canada, what to call the judge depends on the type of court (Federal, Provincial, Family, etc.) and varies from province to province. In Ontario, they do indeed refer to their judges as ‘Your Honour’. In Nova Scotia, Family Court judges are called ‘Your Honour’.