Microsoft Needs To Quit Fucking With My AdSense Scripts

A few months back I discovered that I was being hit with massive amounts of referrer spam. It was a little odd, because the phrases weren’t made to look like I had links coming from some sleazy little viagra or porn site… no, these were made to look like I was ranking for all kinds of spammy phrases on MSN Search. This confused me, since historically referrer search only benefits the site that is made to look like the traffic is coming from. I couldn’t understand why someone would want to make it look like MSN was spamming my logs.

What never occurred to me, of course, was that my logs were actually being spammed by Microsoft itself. As it turns out, they were.

Apparently all of the traffic was originating from 1 IP block, 65.55.165.*, which reverse DNS’d to names like bl2sch1081901.phx.gbl. Doing an IP whois on the addresses returns this data:

IP Location: United States United States Microsoft Corp
Resolve Host: bl2sch1081901.phx.gbl
IP Address: 65.55.165.35
Blacklist Status: Clear
Whois Record

OrgName: Microsoft Corp
OrgID: MSFT
Address: One Microsoft Way
City: Redmond
StateProv: WA
PostalCode: 98052
Country: US

As I looked into it, I found that of course I was not unique. I first discovered the activity on August 18th, and wrote about it on SEO Refugee. As soon as I started to research it I found a thread on WMW discussing the same thing had been started the day before. When asked about it, MSNDude (who apparently at one time anyways was verified as working for MSN) gave this as his reasoning for the behavior:

The traffic you are seeing is part of a quality check we run on selected pages. While we work on addressing your conerns, we would request that you do not actively block the IP addreses used by this quality check; blocking these IP addresses could prevent your site from being included in the Live Search index.

Makes sense… filling our traffic logs with bullshit queries, putting extra loads on our servers, and generally acting irrationally sounds like just the kind of “quality check” most websites need, and as we all know threatening to ban our sites from your search engine if we don’t let you do whatever the hell you want is always the way to go.

Annoying as all that is, however, it is still nothing compared to the real clincher here. This bot sent by Microsoft to attack our innocent websites isn’t just downloading the HTML pages… it’s also downloading CSS files and Javascript, including downloading AdSense code from Google. I included a snippet of my logs from Aug 17th to show what I mean here. You can see where the bot, which is spoofing IE7, and never once downloaded robots.txt, hit the page, then downloaded my local .js file, followed by the external css, followed immediately by a visit from the “Mediapartners-Google” bot… which means that Google thought that this rogue bullshit bot from Microsoft was actually a visitor to my site.

Now, while we cannot know for sure if this will affect the earnings per click (raising impressions causing our CTR to drop), we do know that on AdWords, Google will slow delivery, stop displaying altogether, and raise the minimum bid for those advertisers whose ads have a poor click-through. This means that it makes perfect sense for Google to reserve ads with the better chances of making them money for sites on the content network that have a better chance of the ads actually getting clicked. If this is in fact the case, then Microsoft, in addition to annoying the hell out of me by screwing up my traffic reporting, is also fucking with mine and other people’s income. This is just not acceptable.

I did write to AdSense, but seemed to be unable to get the tech to understand that this was not a normal bot. She kept “assuring me” that hits from bots were not recorded, despite my explanation to her. She said they’d keep an eye on it… but of course would not tell me if in fact a lower CTR would affect earnings per click. One fact she did state I found quite interesting:

Please be assured that we only count impressions when the AdSense ad code is executed by a user’s browser.

Since the code actually has to be executed, then what strikes me as slightly “strange” is the fact that for some reason the screwed up MSN bot sees fit to download and process AdSense code, yet leaves Mybloglog Javascript untouched. Hmmm.

The very day I discovered this, I blocked that entire IP range from being able to hit my site using .htaccess, but it is still trying to get through. Although I blocked it from hitting my site, I cannot block it from hitting AdSense, and apparently it has a bunch of the code snippets for those cached, and seems to be still pulling them. The main 2 symptoms are that my AdSense is still showing inflated impressions (compared to Mybloglog stats, which the bot does not process apparently), and phantom visits from the AdSense bot, where it visits a page it thinks someone hit, when no one did.

I would be very interested in hearing other peoples thoughts, opinions, and experience with this, if they have any.