What Will *Really* Break If Google Switches To AJAX…?

On Friday I wrote a piece on how it looked like Google was testing AJAX results in the main serps. Some discussion followed as to whether, if this change were to become a widespread permanent one, this would affect Firefox plugins that existed (definitely some existing ones would stop working), break some of the rank checking tools out there (they would have to be re-written I’m sure), and even some people asking if it would thwart serps scrapers from using serps for auto page generation (not for long, no).

While those things would definitely be affected in at least the short term, there is a much greater impact from Google switching to AJAX. All of the issues mentioned involve a very small subset of the webmastering community. What actually breaks if Google makes this switchover, and is in fact broken during any testing they are doing, is much more widespread. Every single analytics package that currently exists, at least as far as being able to track what keywords were searched on to find your site in Google, would no longer function correctly.

The reason this breaks is core to the way all of the browsers handle referrer strings, which is how the browser tells the web server how it got to your page in those cases where it does send that information. Sending the referrer string is optional, and can even be turned off, although by default it is on in all of the major browsers that exist today. Analytics programs, whether they are log based (programs that go through the server logs reading the referrer strings) or Javascript based (such as Clicky, the built in tracking in MyBlogLog, and Google Analytics) use that referrer string to determine what it was that that someone actually searched on, assuming that they found your site via a search engine, before reaching your site. For Google, they analyze the querystring portion of the referring url, and look for the “q={keyword}” parameter. So for instance, if you were to find the following url in your logs:

Google search for [troll defense]

then you would know that someone had gone to Google, searched on [troll defense], and then clicked on the link to your site. If you were running some form of analytics program, that program would then register and track that search for you, so you knew what was and what was not sending you traffic from Google. If you run Google Analytics, you can even hack it to get the full referring url and get even more information from the referral.

However, if Google switches over to AJAX none of this will be possible. Unfortunately this isn’t even something that could be accomplished with a simple recoding of the analytics packages either… it would require completely rewriting the browsers themselves in order to track referrals from the new Google AJAX searches. The new AJAX url, as I mentioned in the earlier post, is driven by parameters that come after the hash mark (the number or pound sign in the url). Browsers do not include that data in the referrer string, and it is never sent to the server. Therefore, all referrals from a Google AJAX driven search currently make it look as if you are getting traffic from Google’s homepage itself. Now, while this kind of information showing up in your tracking programs might be quite a boost to the ego if you don’t know any better, and will work wonders for picking up women in bars (“guess who links to me from their homepage, baby!”), for actual keyword tracking it is of course utterly useless.

On Friday I set up a couple of tests to demonstrate this, so I could show the results in the server logs, MyBlogLog tracking, and even in Google Analytics itself. I found a phrase that I was ranking for, that I knew that no one was searching on, [bad neighborhood vandemar], and sent myself some referrals. In every case the method I checked showed Google.com itself as the referring site. You can see this in the server web traffic log:

Web traffic log, Google.com referrer

in MyBlogLog stats reporting:

MyBlogLog stats, Google.com referrer

and even in Google Analytics:

Google Analytics, Google.com referrer

I don’t know if Google has considered these ramifications in making this switch over, or if this testing is something that they are actually interesting in pursuing. I don’t know why they would be testing a change of this magnitude in the first place, though, unless there was a good chance that they would eventually go live with it.

69 thoughts on “What Will *Really* Break If Google Switches To AJAX…?”

  1. Michael, the link was just an example of how you can set a header into a request from the client side. That was all. The point is, IMHO I don’t think Google will go live with these changes.

  2. And Cristi my point was that you can only do that if the page the code is on was generated by the same server you are sending the request to. It’s a security issue. It cannot be done going from one website to the next.

    You may be right though, they may not go mainstream with these changes. We won’t know for sure until it happens.

  3. I definitely don’t have the technical chops to keep in step with these posts so I maybe way off base here. But, wouldn’t you think the folks at Microsoft and those who work on Firefox would be inclined to quickly update IE and Firefox browsers to pass through the info after the hash? Wouldn’t that be priority number one to prevent Google from gaining more strength in the marketplace? After all, knowledge is power and if Google controls the information they continue to increase their power.

  4. How does this affect adwords reporting in google analytics? Will we no longer see which keywords resulted in click throughs for an ad? would this be affected as well?

  5. @Alex – I doubt it, but hypothetically speaking, even if they did it would still require the entire online community to switch to those newer versions in order for the webmasters to see that referrer data again on their end, and a still require a major rewrite of all existing tracking software to utilize it.

    The point is, this change is not a trivial one.

  6. I’ll bet Google will find some workaround for this, especially if it “breaks” their own Google Analytics tool. Either add in an additional URL parameter (which would be in direct contract to their AJAX-experimenting, since it would require a page reload), or attach an extra parameter for any SERP-item followed. Something in line of www.site.com/mypage/?__google_search=my+site+in+google.

    Or perhaps a more ugly solution, having a form of redirect before being sent to your final search-result page.

  7. It seems disingenuous of Google to say that they’re doing it to “provide a faster experience for our users”. There must be another reason.

    I do hope they’re being honest about the fact that they have no intention of disrupting referrer tracking and they are continuing to iterate, because if they do roll this out they are going to break a lot of successful business models.

    However, I suppose we can take some solace in the fact that rolling out this change would in fact break Urchin, because, I agree with Cristi, it seems unlikely that Google would do that.

  8. Michael,

    Which Google Analytics report did your screen shot come from? Was that the Keywords report under Traffic Sources? I just want to check and see if there is something similar showing for our accounts. As you can probably tell, my knowledge on all this is a little limited. And I apologize if this question already arose; I didn’t take the time to read through all the comments.

  9. Melanie, it’s from Traffic Sources -> All Traffic Sources. Google Search referrals look like “google / organic”, whereas referrals from Google that are supposedly not searches look like “google.com / referral”.

    Just so you know though, it looks like they have cut way down on the testing it right now. It might be related to the fact that they were only testing one browser, and Firefox updated last week to a newer version. Or they may have just slowed way down because it drew too much attention. Not sure. You should still be able to see what I was talking about though if you restrict your stats report to Feb 1st through Feb 5th. For that period a little more than 25% of my Google traffic was direct, without keywords passed.

  10. From a technical perspective, there’s really nothing *preventing* Google from passing the actual referrer and search string from SERPs.

    Some may have noticed recently (within the last 2-3 months, I would say) that Google modified their tracking which links you click as you exit their site. As I often do right-click “Copy this link” on SERP pages to bookmark things for my blog, I noticed a lot of them redirected through Google. Checking today, this has been changed, again.

    For a brief period (after I checked again) they were redirecting organic search links through google.com in order to track them for a short period. Now it appears that they are tracking exit links using Javascript (what, no GA?). Look at each organic link on the page’s source:

    onclick=”return clk(this.href,”,”,’res’,’4′,”)”

    I offer a detailed analysis of this internal tracking over at my personal blog, as this in particular is off-topic, but it reminds me of the quick technical solution which lets Google use AJAX and analytics to continue peacefully is:

    – Use AJAX to perform their search using the URL fragment to “hone in” on results (http://www.google.com/#q=roi+tracking)

    – Upon the outbound click, redirect through Google.com without using a referrer-killing HTTP status

    (that is, Google.com servers would NOT use a 3xx HTTP status so the referrer is preserved as the outbound page containing the query string)

    – Everybody wins.

    If Google chooses to kill referrers, and the main source of SEO goodness, then I think it’s best to rethink their “Do no evil” mantra.

  11. And one more note for the conspiratorial minded: If Chrome chooses to pass the “fragment” to browsers, this would likely break a variety of sites (web servers, specifically) which don’t know how to handle them.

    It’s against the standard and I have seen many a web server which just plain break when given anything outside of a normal URL (Java web servers pop to mind, in particular.)

    So, would Google risk having their browser break on some web servers? Doubtful.

Leave a Comment

*