Using my open source website statistics program, Obsessive Website Statistics, I monitor my traffic on a regular basis mostly to see if there are any links from anywhere interesting. Well, lately I’ve been noticing a large number of referrer links from search.live.com with random single-term search terms in them. So I looked at the hostnames and noticed this (this is just a sampling):
The IP’s and non-standard domain names all belong to Microsoft.The agent is always the same. It always browses maybe one or two pages deep, possibly grabbing some CSS files in the process. Its definitely a parallel bot of some kind, since its all from the same IP ranges, and they all have similar queries. Additionally, it NEVER checks robots.txt .
Researching this on the internet (using Google, of course) shows I’m not the only one affected by this. Apparently MSN has claimed that its just a “quality test”, but thats BS. They’ve been quality testing since August 15 for me, in that case.
My first reaction has been to agree with Josh Cohen, who posted in his blog that he thinks MSN is doing good old-fashioned referrer spam to try and entice people to use Live Search since the normal methods of getting people to use it have failed.
Its certainly enticed me to click through a number of times, only to discover that my site isn’t anywhere near the top of the listings for those terms. In fact, originally the links weren’t even resolving to proper pages, and it would give me an error when I actually got to Live Search. Go figure.
If anything, this has convinced me to NOT use Live Search. Thank you Microsoft, for making my log files a little more annoying. Microsoft needs to stop doing this, and do a “quality check” on their own procedures.
Sampling from my log files containing these accesses: