Last month I blogged about the fact that I had noticed that Google was playing around with delivering the SERP’s via AJAX. I pointed out that due to the way that referrers work, using AJAX to generate the pages would cause all traffic coming from Google to look like it was coming from Google’s homepage instead of from a search. This means in turn that analytics packages, including Google Analytics, would no longer be able to track what keywords searched on in Google were sending traffic to the webmaster’s websites. There was a bit of a buzz about it, and Google seemed to stop the testing shortly thereafter. Google’s only reply on the subject was “sometimes we test stuff”, to point to a post from three years ago that also said, “sometimes we test stuff”, to say that they didn’t intend to break referrer tracking, and that was it.
Shortly thereafter, the tests appeared to have stopped. People stopped thinking that Google was linking to them from their homepage, and in very short time the buzz died down about it.
Yesterday afternoon, someone pointed out to me that the subject came up again during Matt Cutt’s keynote address at PubCon South. According to Lisa Barone’s live blogging efforts the conversation went something like this:
Ok, fine. So they didn’t know about the referrer issue (and didn’t give me credit for pointing it out to them before it was more than just a test, *cough* *cough*), so they stopped until they can figure out a way to fix it.
Less than 1 hour after reading that, I happened to notice an over-abundance of url’s in the serps that were getting redirected though Google’s url redirection service. For those who are unaware of what I mean by that, it is a tool that Google uses to enhance their own behind-the-scenes tracking of user behavior. You can see it most consistently when Google displays sitelinks. They redirect the clicks through their own tracking mechanism first, so that they can determine how many people are actually using those extra links. Instead of going directly to the page in question, it goes through a link like this:
If you view one of those links using a header detector, you can see that after doing whatever tracking they do on their backend, they then redirect the user via a 302 Redirect onwards to the final destination page:
User-Agent used to fetch header:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008051206 Firefox/3.0
HTTP/1.0 302 Found
Content-Type: text/html; charset=UTF-8
Date: Fri, 13 Mar 2009 15:44:42 GMT
Occasionally you will see these Google redirects in the normal serps as well, although usually not. The thing is, I was seeing them on every search I performed. It struck me as odd, until I suddenly realized that every search was being done via AJAX:
Here’s the problem, Google. That will not fix the referrer issue, which is what the issue is with every non-Google analytics package that exists. Without that, then the traffic coming from Google cannot be accurately analyzed (unless, of course, the analytics program has access to whatever it is that Google’s redirect script is recording). For one, a 302 redirect passes on the referrer of the original page, not the one of the tracking script, and for another there are no unencrypted keywords included in your tracking urls. It doesn’t take pushing the test to the live servers to figure this out, either. The engineers had to have known this beforehand. This means that if Matt was correct, and Google did indeed stop the testing until they could make it work with analytics, then the only analytics package they were worried about AJAX serps working with is Google Analytics.