As “pure webspam” has decreased over time, attention has shifted instead to “content farms,” which are sites with shallow or low-quality content. - Matt Cutts

Whoa. This, especially coming from Matt Cutts, is huge. For those who don’t know, “content farms” are organizations that generate websites composed of large amounts of low cost “fluff” or filler content, with little to no regard to quality. The content is generated not based on having information and the desire to share it, but rather in response to queries that might get typed into a search engine, and are built for search spiders rather than human consumption. They include companies like Demand Media, Mahalo, and Associated Content.

Historically speaking, Matt has pretty much refused to come right out and say that these content farms were indeed spam, despite the fact that they clearly violated Google’s quality guidelines:

Doorway pages are typically large sets of poor-quality pages where each page is optimized for a specific keyword or phrase… Google’s aim is to give our users the most valuable and relevant search results. Therefore, we frown on practices that are designed to manipulate search engines and deceive users by directing them to sites other than the ones they selected, and that provide content solely for the benefit of search engines. Google may take action on doorway sites and other sites making use of these deceptive practice, including removing these sites from the Google index. - Google Webmaster Tools Help

Regardless of the very clear wording of their policies, Google has to date not banned any of these content farms for their violations. In fact, quite the opposite – Matt has in the past even defended these sites, and in Mahalo’s case at least given warnings to them which he then allowed them to ignore. He alluded to the fact that one of the algorithm updates from last year, Mayday, was supposed to help filter out “really kind of lower quality” sites, and many people thought he must be talking about content farms back then, but alas that turned out to be a bust. So when he comes right out and says, hey, you’ve waited long enough, now we’re going to target content farms for reals, y’all, then yeah, that’s a Pretty Big Deal.

Now, Richard Rosenblatt, the CEO of Demand Media, may be may be in denial about his company being a content farm, but that definition has existed for quite some time, and regardless of what you call it low quality content built specifically for search engines is in violation of Google’s guidelines. However, he still persists in his belief that as long as you can get some people to call it something else, his “partnership with Google” will keep them protected regardless of what happens:

This is why our partnership with Google makes sense. 1) We help them fill the gaps in their index, where they don’t have quality content. 2) We’re the largest supplier of all video to YouTube, over two billion views and 3) we’re a large AdSense partner. So our relationship is synergistic, and it’s a great partnership. And it’s a partnership that we’re excited to continue to expand. - Richard Rosenblatt, attempting to give Google’s PR team a heart attack

I am guessing that Mr. Rosenblatt missed the section in Matt’s post where he very specifically discussed the fact that no special partnerships would protect the content mills from these changes:

One misconception that we’ve seen in the last few weeks is the idea that Google doesn’t take as strong action on spammy content in our index if those sites are serving Google ads. To be crystal clear:

• Google absolutely takes action on sites that violate our quality guidelines regardless of whether they have ads powered by Google;
• Displaying Google ads does not help a site’s rankings in Google; and
• Buying Google ads does not increase a site’s rankings in Google’s search results.

– Matt Cutts, being crystal clear

Then Friday rolls around, and Matt announces that these changes already happened earlier in the week. If you didn’t notice any changes, then that’s probably because, according to Matt, less than half of a percent of queries would show any perceptible ranking differences. If you didn’t notice any changes in queries involving content farms, well… as near as I can tell that is because there weren’t any. In fact, in his announcement post Matt doesn’t even use the phrase “content farms” at all, and instead only discusses that the net effect of these changes is that in cases where content was scraped, searchers are more likely to see the original content first. He then thanks Jeff Atwood (one of the ones who wrote a story discussing Google’s decline in quality that had a large audience) and Stack Overflow’s team (a site that Jeff co-founded) for their feedback. A few people asked about the omission in the comments, but as of yet anyway Matt has not replied to any of them.

As to the results themselves, for the most part I am seeing what I was seeing before, so that “less than half of a percent” doesn’t surprise me. If you search for [mcdonalds coupons] the #1 site is still a Mahalo page that doesn’t actually have any coupons on it, and very little original content. If you search for [mcdonalds free salad coupons] you get a different Mahalo page that does actually have a picture of a coupon on it (good only in Canada, and expired in July 2010, however), and if you search for [mcdonalds happy meal coupons] the second listing is a Mahalo page, again with no coupons on it. These pages are filled with riveting dialog, such as the section labeled “McDonalds Happy Meal Coupons Coupon Policies,” which states:

The policies for McDonalds Happy Meal coupons may have certain restrictions and these might include not being able to combine discounts or limiting the period of use. Make sure you read and understand the instructions listed on the coupon carefully to ensure that you know when the coupon will become valid and when it will expire as well as what special restrictions apply. Also included in this information will be which product or products the coupon can be used to purchase. Insuring that you understand the coupon policy can help you to avoid any mistakes during the checkout process. - Content Mahalo actually paid for

Seriously?

It’s not just Mahalo, of course… type in [how to reset your blackberry] and you will find ranking just fine a page from eHow that is nothing more than the phrase “hit alt+right-shift+delete” wrapped in light, fluffy filler content. I also still see queries where the duplicate content outranks the original, such as the copy of a Wikipedia page that ranks #1 for [elvett semic]. The changes, whatever they were, truly are barely (if at all) perceptible. The change was so small that one of Matt’s readers asked, “I’m wondering why announce it if you’ve gotten the feedback and the algorithm update would presumably be of such little consequence that no one would likely notice or comment on it unless you told everyone.” Indeed, why make such a big deal out of something when almost no one can tell the difference?

To answer that you need to take a look at exactly what it was that did change. When I search in Google now for questions that were asked on Stack Overflow, at least for the queries I checked, I now see SO ranking instead of sites that scrape their content. This is of course how it should be, and the main concern that the people from that community were bitching giving feedback about to Matt. Stack Overflow is, as I mentioned, the site that was co-founded by Jeff Atwood, who is the author of the much quoted post that generated quite a bit of buzz about Google’s decline in quality. Many of the frequenters of Stack Overflow are also regulars on Hacker News which (not so) coincidentally Matt decided to hold a good portion of the discussion about these changes, both before and after they were implemented. While the HN and SO communities in and of themselves might be tiny compared to the web as a whole, the fact is that their voices do carry within the online community. Start buzz there about Google showing quite a bit of improvement and it has a very good chance of spreading, even if the data set demonstrating that is overall quite small. Add to that the fact that Richard Rosenblatt, CEO of Demand Media knows that the changes aren’t targeted at his company (and when asked if Google had discussed the changes with him, replies “I can’t comment on that.”), and then toss in Jason Calacanis’s ingratiating comments on Matt’s blog post about the changes going live:

It was clear that Mahalo was getting grouped into the “content farm” space… - Jason Calacanis

No kidding? Really? Past tense there, eh Jason?

So Matt loosely ties the concepts of “content farms” and “scrapers” together in a blog post on the official Google Blog, and claims that they are taking action against them. He then announces a change that appears to only affect scraper sites, and furthermore only those scraping a specific dissatisfied community, publicly thanks that community for their help, and then doesn’t mention the phrase “content farm” again. Even though the changes were practically non-existent, there is a good chance that the overall impression from those who don’t look too closely is that action was indeed taken, and that if what were formerly referred to as content farms are still ranking well, then obviously they must be there for a reason.

From a strategic standpoint it’s actually rather clever. If I were Google and I needed to conceal special relationships I had with companies (especially if I was thinking that the FTC might want to get involved in my business) then I too would probably try very hard to sway the public opinion about the labels attached to the sites those companies owned, and shift the focus to something I could fix without caring about the damage, and then crowd source a tech community to help spread the impression that things were better. Most people probably won’t even pay enough attention to notice.

(click to view original)

Personally, I’d like to see Zedo up its standards for the type of ads it will accept. This type of junk shouldn’t be allowed. - Danny Sullivan

He’s right, too, the ad networks should be policing this type of deception, by all means. Matt Cutts, Google’s head of the web spam team, agrees. He tweeted about the story, and also commented his take on the matter in the post itself:

My favorite part of the disclaimer for those type of sites is “This website, and any page on the website, is based loosely off a true story, but has been modified in multiple ways including, but not limited to: the story, the photos, and the comments.”

Oh, so I can trust the website except for the story, photos, and comments? In other words, the entire website?

And if you read the disclaimer carefully, most of these sites promise a “free trial” with $1.95 in shipping, but actually set your card up with a recurring subscription. The “one weird old tip” ad that I clicked from the L.A. Times mentioned this in the fine print: “If you do not cancel within seven (7) days of the date that you enroll in the Program, we will charge the same card you provided at enrollment the non-refundable one-year membership fee of $149.95″. Then they also start charging you $12.95 a month. Grr. – Matt Cutts, on deceptive “flat belly” ads

Grr, indeed.

Danny also mentions in his post about how “The ad, unlike Google’s ads, doesn’t report what ad network is delivering them,” which if they did would be a form of disclosure. And Danny is right… except for one thing. Danny derived the fact that the ad was being served by Zedo by examining the url. However, if you view the source on the LA Times article and go to the spot on the page where the ad is showing, you don’t see the Zedo ad network code. The ad itself is being generated by Javascript that is being pulled from yet another ad network:

The actual ad network that the LA Times has a relationship with, and the ones responsible for what ads show on their site, is Doubleclick. And who owns Doubeclick, you might ask? As most of you probably already know, Google does, since they bought them back in 2007 for $3.1 billion. So obviously not all of the ads Google delivers disclose what network they are from.

It gets better. AdSense, Google’s flagship advertising network, serves what are known as “contextual ads”, where in theory the ad targeting is based on the context of the page contents where the ad blocks are placed. Danny uses AdSense on his site, with one of the blocks being at the very top of the page. Due to the various feeds in the sidebar, the content of the article, and the title, “Acai Berry” is mentioned 8 times on that same page. Therefore it is only natural, of course, that this is what we see when we look at the ads being served on the top:

(click to enlarge)

Now, can you guess where that ad leads? That’s right:

(click to enlarge)

It’s a fake news site identical to the one Danny is discussing, with the same text, layout, and even images embedded in the “story”, with the only variation being that the one Danny landed on is “News 7″, and this one is “News 8″.

What makes this story particularly interesting is that recently Matt Cutts visited Washington D.C., lobbying the FTC about Google’s integrity, trying to convince them that they don’t require government oversight, and how they could be trusted to police themselves. Google also happens to be in a very unique position to help clean up these kinds of abuses. Not only could they pull these ads from their own vast array of properties, and require their third party partners to do the same, but they could also warn publishers who use networks that continue to promote scams that their sites rankings could suffer, in the same way that they have punished websites in the past for what they said was deceptive marketing, in the form of undisclosed paid links. Instead, they themselves appear to be participating in the problem, not the solution.

So, Matt, are you willing to back up your testimony to the FTC about Google’s integrity, and lobby within your own company to help eradicate deceptive marketing from the web? Do you feel that websites that allow deceptive advertising to be shown on their sites should have their trust revoked?

Each time it happens Matt’s defense of Mahalo spamming Google just gets more perplexing. In this latest round he started by saying that his job was not to have knee jerk reactions, as if Mahalo hadn’t already established a pattern of spamming over a long period of time, and that Matt is pretending he hadn’t already had a talk with Jason and told him that if he didn’t raise the bar with his site that Google would take action on Mahalo. From there it got even weirder – Matt looked at the linkfarm and basically told me that a) he didn’t care as long as it wasn’t passing link juice, and b) he’s the only one who could tell if that was the case.

I could have sworn that it was if you were caught trying to spam you were penalized, and you couldn’t get the penalty removed unless you promised not to do it again. Now, where did I get such a crazy and wild idea? Oh yeah, I remember now… it was from Matt Cutts:

Now we come to the heart of things: what goes into a reinclusion request. Fundamentally, Google wants to know two things: 1) that any spam on the site is gone or fixed, and 2) that it’s not going to happen again. – Matt Cutts on the bare essentials of a reconsideration request

The reasons Matt gives out for defending Mahalo seem to be getting more and more creative (even if not more believable). Jason’s, on the other hand, are the same old song and dance he has been spouting since I first called him on his bs and demonstrated that the vast majority of his site was nothing more than empty, auto-generated pages. On Thursday’s post, before he started to lose it with his “fuck you losers, I’m rich” tirade, Jason made this statement:

We have humans write pages of at least 300 words. We don’t index 99.99% of pages with < 300 (it would have to be something unique), and we police the system to get short pages up to 300 words within 30 days. - Jason Calacanis, 4 days ago

Orly? Let’s take a look at those claims, shall we?

The Mahalo coupon pages are about the crappiest pages I have found on the site. When I was doing my initial investigation I stumbled across quite a few of them. My guess is that [{brand} coupon] generates AdSense blocks with a decent eCPM since they are, after all, “targeted” pages. None of the Mahalo “coupon” pages actually have any coupons, which of course means that the end user is much more likely to click on one of the ads when they land there, and more required clicks does means a poorer user experience. What content these pages do have is fluff text that gives ample opportunity for Mahalo to link back to itself, and have spammy signals that are easy to spot like when there are near-identical versions of the same topic page, usually by doing one page for “coupons” and another for “printable coupons” (and no, there is nothing to print out on those pages either). Therefore i picked those as where I would look first to point out, yet again, how Jason was simply pulling these claims out of his ass with no supporting truths behind them.

Digging back into my old data, from March 13th, I was able to determine that from the day the site started adding content up until that point in time Mahalo had amassed 2,655 coupon based pages. When I re-scanned and looked this time I found that there was now 16,601 of these pages. That is a huge increase for 3 months, and a ton of content to create uniquely, even if you ditch quality altogether. Mahalo currently only has a grand total of 90,494 of actual pages on that side of things, so that means 18% of the site is made up of “coupon” pages – and by that I mean coupon pages that don’t actually have any coupons on them.

What’s more, it actually looks like there is a chance that 9,932 of those pages were added last week, over a 3 day period. How the hell do you get writers to create 9,932 pages of even crappy content, all about coupons, in only 3 days?

As I started looking into it I suddenly understood… they didn’t just ditch the quality to create those pages, they went ahead and ditched the content, yet again. I checked over 30 pages, and time after time I found what I found was auto-generated pages that were nothing but ads, affiliate links, and scraper feeds.

http://www.mahalo.com/1800pools-coupons:

(click to view full page screenshot)

http://www.mahalo.com/tigerdirect-coupons:

(click to view full page screenshot)

http://www.mahalo.com/topnotchcare-com-coupons:

(click to view full page screenshot)

Most of the pages I checked had the affiliate links provided by Savings.com, and most linked to the same two questions pages: one discussing the Outback coupons page, and one discussing “grocery coupons”… and in every case neither question had anything to do with what the actual “coupon” page was supposedly about:

For the pages that did not have Savings.com affiliate feeds on them it was because they were using as keywords the names of sites that wouldn’t actually be Savings.com publishers, like GBB.org and RLS Forum. It looks like Jason somehow got his hands on a list of sites that for some reason or another looked like they might have offered some sort of coupon. These were then dumped into the database in the form of pages, and were then checked to see if they matched up with the Savings.com feed. If they did, great, if not that’s ok too, they still had AdSense on them – despite the fact that putting AdSense on pages without actual content is a direct violation of Google AdSense policies:

That’s ok though, I am sure Jason doesn’t care that he is risking the bulk of the site’s revenue stream by violating the terms of the program, since it looks like the AdSense team is giving him just as much of a pass as the spam team is.

In addition to the pages simply being devoid of content, Jason also uses the tactic of creating near-duplicate versions of some of these pages in order to get the most out of the long-tail phrase variations:

http://www.mahalo.com/1and1-coupons
http://www.mahalo.com/1and1-internet-coupons
http://www.mahalo.com/1and1-web-hosting-coupons
http://www.mahalo.com/1and1affiliate-com-coupons

Let’s look at Jason’s statements again…

We have humans write pages

Well, no. You have humans write some pages, but an assload are still auto-generated. In addition to the ones shown here, Google also says that you still have 13,200 pages that you scraped from Wikipedia in their index:

Adding the above auto-generated pages in with the Wikipedia ones, that means that at this point an estimated 33% of the Mahalo content pages are scraped or auto-generated, and that’s just the stuff that’s easy to find. Yay footprints.

of at least 300 words

Again, no, even on the human generated pages that is not always true. Take a look, for instance, at the 1and1 page on Mahalo.com that all 4 of the above coupons reference:

Including words of 3 letters and less that page still only has 212 words of human generated content on it. I also pointed out last week that some of the Wikipedia scraped pages remained thin, such as the one on “The Alice B. Toklas Cookbook”, which has only 261 words on it.

We don’t index 99.99% of pages with < 300 [words]

Bullshit. Not one single one of the pages I examined had a “noindex” tag on it, or was blocked by robots.txt. In fact, just the opposite – every single one of them was pushed to Mahalo’s sitemap, to make it easier for Google to find (and index) them.

we police the system to get short pages up to 300 words within 30 days

Again, bullshit. The 1and1 page has been that way since at least March 11th:

And the Alice B. Toklas one since March 12th:

So Jason, please, enough with the bs. Quit claiming stuff that simply is not true, especially when it’s so damn easy to disprove what you say. I still have no idea why it is that Matt Cutts is choosing to ignore your spam, but to the rest of us it’s as plain as day. And no, Jason… going in now and trying to clean it up in no way changes the fact that you spammed in the first place.

These sites usually have no original content and may be cookie-cutter sites or templates with no unique content. – Google Webmasters Tools Help, on sites Google does not like

These sites that Mahalo has started churning out, all that were apparently created just this year, would appear to be the AdSense version of the classic “thin affiliate” website.

I showed Matt Cutts the link to the search itself, and asked if he thought that the list of sites being returned looked spammy to him. His reply?

So, yet again, for some inexplicable reason the head of the Google Web Spam team appears to be defending Jason Calacanis. Despite the numerous posts that clearly demonstrate that Jason Calacanis is spamming Google, Matt is saying that he needs proof that these new sites are spammy. Pretty much anyone else in the industry can tell at a glance what is going on, but Google’s foremost expert on the subject of spam still needs help seeing it.

That’s fine. Let’s go ahead and take a deeper look at what is happening behind the scenes with these sites. Here’s one of the new sites that deals with cooking, cooking-questions.com:

336 pages indexed there. So, was the “cooking” topic on Mahalo.com not covered then…?

6,680 pages currently on Mahalo Answers, all about cooking. Looking at that list you can tell from the first two listings that Mahalo has one category for “Cooking Recipes”, and a completely separate category for “Cooking and Recipes”, whereas most non-spam directories, blogs, etc. would simply have picked one or the other. If they are struggling with diversity on the main site, how is it adding the same topic to an entirely new site is going to help?

Just for the record, the new, smaller site also seems to see the need to have both of those nearly identical categories as well:

Also, in case you’re thinking that maybe it’s the individual questions themselves on these new sites that are “diverse”, one of the questions on the new site is “how-do-you-know-when-corn-on-the-cob-is-fully-cooked”. Mahalo.com already has 10 pages on corn on the cob, 3 of which are: “how-long-do-you-cook-corn-on-the-cob”, “how-many-minutes-do-you-think-is-the-perfect-time-to-cook-corn-on-the-cob”, and “what-is-the-perfect-amount-of-time-to-cook-corn-on-the-cob”. Yeah, that’s diversity for ya.

Let’s look at some of the other sites, see if maybe that first one was just a fluke. The new Mahalo site for Star Wars, starwarsanswers.com:

308 results. So, is Star Wars not handled on Mahalo.com then?

1,200 results, so obviously this isn’t an example of “better diversity” either. Similar results for their Oklahoma City site, oklahomacityanswers.com:

177 pages, and yet again, not new subject matter:

632 pages. On Oklahoma City questions. How did people manage to come up with that many questions about Oklahoma City on a site that hardly anyone ever actually goes to on purpose?

The answer, of course, is that they didn’t. On the new site there are exactly 11 actual questions as of this writing. The other 166 pages are (mostly empty) category pages, member pages, and other fluff that each of these “cookie-cutter sites or templates with no unique content” come with by default. Since this is all crap content then, why would Calacanis even bother with them? There is little to no interest in many of these subjects… what game could Jason possibly be playing here?

I know! Let’s all play:

Back in February I discussed how the internal pages on Mahalo.com get almost no natural links. Almost all of their PageRank (and thus ranking power) comes from either employees linking from their blogs or it comes from scrapers. With hundreds of thousands pages to support, however, relying on that kind of sketchy link profile has a good chance of not panning out in the long run. Someone must have pointed this out to Jason, and now these mini-sites are his solution. If you can’t build enough quality pages for people to want to link to you naturally, build sites and link to yourself. Look at, for instance, the cooking site’s question on “What is the most common way to make Angel Food Cake?”:

Within just the question itself, including the title, there are 14 links, all pointing back to Mahalo.com topic pages. If we examine the link profile for those pages being linked to, we see that these linkfarms that Jason is putting up provide the majority of their link juice. For example, the first page linked to is Malao’s page on “cake”:

(click to enlarge)

Of the 10 links listed, 4 come from scrapers, one is a url shortener that is actually on a Mahalo.com page, and the other 5 all come from Jason’s linkfarm: cooking-questions.com, parenting-questions.com, and foodiequestions.com. You can see the same is true throughout when you check the other links in the question, like the ones to the Mahalo pages on “chocolate” [link:http://www.mahalo.com/chocolate -site:mahalo.com] and “baking” [link:http://www.mahalo.com/baking -site:mahalo.com].

Some of the pages being linked to don’t exist any more, since Jason did go in and delete some content in response to a good talking to Matt Cutts gave him a couple of months ago… but that’s ok, the links are there just in case they ever decide to build those pages back out.

The embedded links aren’t the only ones on these sites, either. You also have the sidebar links pointing back to Mahalo:

Of course all of these little sites need some link juice themselves, in order to stay indexed and pick up some long tail rankings, which in turn with garner them some scraper backlink juice of their own. Jason’s got that covered too. For some of the pages he has redirected old pages from Mahalo.com, which acts as a reciprocal link exchange between Mahalo and the new sites. On iphoneqna.com, he has added a “Recommended Q&A Communities” block to the sidebar, which adds in an extra level of interlinking to the equation:

And of course there’s always the fall back tactic of Jason simply linking to some of these sites from his personal blog and Tumblr account to give them a little extra kick.

Like much of what Jason does, nothing in these tactics is new. Google has directly addressed the non-acceptability of this type of link building for quite a while now:

… some webmasters engage in link exchange schemes and build partner pages exclusively for the sake of cross-linking, disregarding the quality of the links, the sources, and the long-term impact it will have on their sites. This is in violation of Google’s webmaster guidelines and can negatively impact your site’s ranking in search results. – Google Webmaster Tools Help page on Link schemes

It is spam, pure and simple. There is no added user experience, no diversity, and no reason for all of these sites, including Mahalo.com, to not get banned from Google.

What’s your take on “addon domains”? Does Google penalize someone for having one or more addon domains on their main website, (or if they’re self hosting)? e.g. 2, 5, or 10 all coming from the same IP address, would that be bad? – Land Lubber, CO

Matt responded with the following:

Land Lubber, I live in a golden castle in the sky, and I have never had to use shared hosting in my life. Furthermore, I never listen to silly seo rumors either, so I had no idea people were even wasting time on such ridiculous ideas. To be honest, I have no idea what an “addon domain” is, even though you spelled it out for me as being multiple domains on the same hosting account. Therefore, I am just going to make something up and talk about that instead. – Matt Cutts… well, sorta, interpreted

Here is the actual video:

My take? Obviously based on Matt’s response as long as you are not doing it for spammy purposes Google doesn’t give a rats ass if you have more than one site on the same account. Hopefully his lack of caring (or knowing) about it clears that up.

Update: Here is a link where Matt dispelled the myth that multiple sites all hosted on the same IP would be an inherent issue with Google all the way back in 2006:

Myth busting: virtual hosts vs. dedicated IP addresses

Matt made similar statements when he was interviewed by WebProNews and the topic came up:

How do I make sure that I am returning the highest quality content, stuff that’s really useful for users, whether it’s editorial discretion, unique content user generated content, you know, stuff that’s not available anywhere else, versus just something that’s scraped, or duplicate, or really kind of lower quality. – Matt Cutts, explaining how not to get penalized by the Mayday Update

During the keynote, in response to Matt’s explanation of what Mayday was supposed to accomplish Danny Sullivan indicated that he hopes that this update will help filter out results from content mills like Mahalo. Mahalo certainly sounds exactly like what Matt was describing, as I have written about in the past.

So, did the Mayday update actually accomplish filtering out this “low quality” content from the search results? I went back and checked some of Jason Calacanis’ spam pages on Mahalo that I had blogged about in the past. Unsurprisingly enough, most of the pages I checked were still ranking just fine in Google. What was slightly unexpected, however, was what the listing looked like for one of them, [need for speed walkthrough]:

Notice the lack of a snippet for that listing in the search results? That is because due to my earlier write-ups about Mahalo and Google, in an attempt to keep up appearances with Matt Cutts, Jason had the team move a bunch of the pages that I wrote about into a directory named /stub, and then blocked that directory via robots.txt. When Google encounters blocked content that has enough link juice it lists those pages, like you see here, as url-only. What Google usually doesn’t do, however, is actually rank those pages well in the search results.

Thinking that maybe it was a fluke, and that perhaps that particular listing just had some extra ranking power because I had blogged about it before and therefore it gained a few extra links, I delved further. This is just a sampling of what I found:

[fallen angel walkthrough] (#4)

[jack keane walkthrough] (#7)

[how to plan thanksgiving dinner] (#7)

[allheart coupons] (#11)

[adult friend finder coupons] (#1)

Hm. Maybe the key is that if you want your duplicate, low quality, spammy content ranked then all you have to do is block it with robots.txt…?

Obviously that is not the case, and anyone who understands at all about how these things work will recognize that statement as ludicrous… but just to be sure, let’s look at some of the non-blocked content Mahalo pages are currently ranking for:

[julmust] (#7)

[The Alice B. Toklas Cookbook] (#11)

These Mahalo pages being returned are both examples of the many pages on Mahalo.com that are nothing more than content scraped directly from Wikipedia:

http://en.wikipedia.org/wiki/Julmust
http://en.wikipedia.org/wiki/The_Alice_B._Toklas_Cookbook

Note also that Wikipedia identifies some of their content as being “stubs” (the Alice B Toklas Cookbook page has a mere 261 words of content, including numbers and “a”, “and”, and “the”), but Mahalo is fine with presenting that exact same content as non-stub for whatever reason.

So, Danny, sorry… it looks as if Google did not achieve what it reportedly wanted to do with this latest update. It looks like both the low quality and completely duplicate (or even non-existant) content on Mahalo.com continues to rank. Maybe next time.

Let’s ignore, for the moment anyways, that for absolutely any other website on the internet with the evidence that I presented the process would most likely entail the webmaster proving their innocence rather than the person reporting the paid link proving that they are guilty, and that they would have to do so after they actually got banned. Before I get started, keep in mind… whatsyourconundrum.com is not Conundrum Wine’s main website. Their actual website is http://conundrumwine.com/. The site that is being linked to from Mahalo is a marketing device, a link or PageRank funnel, something that acts as an intermediary link bait, or simply to increase brand recognition. This is actually a great way to help increase your exposure without having to put gimmicky items on your company’s professional website. That main website is not hosted on Mahalo.com’s servers.

Now, Matt is correct. The whatsyourconundrum.com website is indeed hosted on Mahalo servers. Without even looking at the IP address, we know by looking at the whois record the dns servers are ns1.mahalo.com and ns2.mahalo.com:

The reason, I am guessing, is that apparently whatsyourconundrum.com is powered by a white label version of Mahalo’s Answers. What we also see in the whois record is that the domain is owned by Camus Vinyards, and that their administrative email is domainadmin@caymus.com. Following that hint, we see that the caymus.com domain is owned by Caymus Vinyards:

Caymus.com is in turn being hosted on ewinerysolutions.com servers. Ewinerysolutions.com is being hosted on their own servers, so the dns trail stops there. If we go to the eWinery Solutions website, we can see that what they do is offer marketing services to wineries:

The most profitable and fastest-growing segment of the wine industry today is the consumer-direct channel. eWinery Solutions offers you the freedom to market your wines easily and effectively by creating a one-on-one dialogue with your best customers.

Jot on over to their Portfolio page, and we see that Conundrum Wine is in fact a client of eWinery Solutions:

Now, does eWinery offer their services to Conundrum out of the goodness of their heart? Somehow I doubt it. Did Jason agree to lease and host the white-label version of Mahalo Answers to them because of all of the warm fuzzies he knew it would give him? Of course not. In fact, he even refers to this deal as being Mahalo’s first “client”:

Whether the header link was part of the original deal negotiated to set up whatsyourconundrum.com, or a separate cash transaction, the fact remains that this was not a merit based link. It has it’s roots in a commercial transaction that most likely occurred between someone at Mahalo and the people doing Conundrum’s marketing. What I also can’t say is whether or not the clean sitewide links pointing to whatsyourconundrum.com on Jason’s private blog was part of the deal or not… but there are 5 of those links there as well, under the heading of “Daily Reads”:

Hopefully Matt won’t ask me to get all ninja and obtain a copy of the actual invoice… that might prove a little problematic for me.

While Google will ban smaller websites from their search results or from AdSense on a whim, usually it takes heavier coverage for bigger players to get hit. Like, for instance, when Google Blogoscoped outed BMW for spammy doorway pages. The story spread relatively fast, and Google wound up banning BMW for a short period of time. So when someone has a “special” relationship with Google, as Jason appears to have, and keeps getting second (and third, and fourth, and fifth…) chances to clean up their act, yet continues to snub their nose in Google’s general direction, it makes one wonder. Google has to be at least somewhat concerned that someone in the mainstream media will eventually notice and start to ask why someone like Jason would continually be allowed to get away with this stuff. Considering the unfairness and lack of impartiality of letting a site like Mahalo slide while punishing so many smaller sites for lesser offenses, my guess is that Google doesn’t actually want to discuss their reasons behind ignoring it. And so, each time Google does nothing, Jason decides to push things a little more.

This time it looks like Jason has decided to go ahead and violate the rules closest to Matt Cutts heart. While the layout I am showing here will change with time, since the header contains rotating articles, currently if you go to Mahalo at the top of every page (on the non-Answers side, anyways) you will see the following block of stories that Mahalo is highlighting (usually this area contains trending or hot news items):

(Click to enlarge)

While technically speaking just by looking at them there is nothing to distinguish one of those “featured stories” from another, the one that doesn’t actually belong is the third one in, with the caption “Best Pickup Line Ever?”. The reason that one is different from all the rest is simple… it’s not a Mahalo featured story at all, and has nothing to do with anything going on in the news. It’s an ad. It is a paid link that Mahalo.com sold, one that leads to a site built to market a wine company. There is nothing visual to distinguish or disclose that as an ad, and if we view the source of the page…

(Click to enlarge)

… we can see that there is nothing machine readable (ie. nofollow attribute) to distinguish it as an ad, either.

Matt Cutts has been very, very clear on his take on sites that sell links that pass PageRank, or ones that don’t disclose that they are in fact ads: they are spamming. No if, ands, or buts about it, they deserve to get punished. In fact, he has even gone to far to state that in his view undisclosed paid links violate FTC guidelines.

So, Matt, recently you put out a call for link spam reports, including “paid links that pass PageRank”. Really, though, is there any point in reporting Mahalo to you? Are you going to actually take action, or, like you have done with Jason’s spam in the past, will you continue to simply look the other way? Any other site would be faced with a penalize/ban first, make nice nice with Google later. Hell, with the BMW site you penalized them after they cleaned it up, just to make an example. I get the strangest feeling, though, that won’t happen with Mahalo…

Can you at least say something about this issue…?

Update: in response to a comment below and a question posed by Matt Cutts about what makes me believe that this is indeed a paid link:

The Mahalo Paid Link Evidence Trail.

Ok Jason, we get it, you’re desperate. But stealing content from Wikipedia in order to replace what you deleted? Come on!

I am flipping through Mahalo.com today, just seeing if you’re keeping your word or not, when all of a sudden I notice this huge amount of pages with odd names that somehow I missed before:

http://www.mahalo.com/cgs-20625
http://www.mahalo.com/cgs-9896
http://www.mahalo.com/cp-154-526
http://www.mahalo.com/daa-1097
http://www.mahalo.com/daa-1106

These are all nothing more than content stolen from Wikipedia. Your version:

Wikipedia’s version:

You even hyperlinked the same internal linking scheme to the same topics Wikipedia does, regardless of whether or not those pages exist on Mahalo.com. How the hell can you claim this is original content when it is nothing more than cut and paste? I mean, wtf, you just claimed that Wikipedia is nothing more than a free for all a little under 3 weeks ago… does the content somehow take on some magical value after you scrape it and host in on your servers, trying to pass it off as something one of your users wrote? THIS is the “our users build it” content that you were referring to?

News flash, Jason… your users and Wikipedia’s users are not the same people. You lumped Squidoo into the same category back then as well. If I look close enough, will I find content stolen from them too?

Now, to be fair, maybe this content existed all along but was much, much less noticeable when you had all of those other pages of fluff in there, but now that you have deleted 78% of that side of Mahalo, these scraped pages are practically impossible to miss. None of the pages I looked at were actually indexed in Google, but it looks like at least 27,900 of them currently are:

(Click to enlarge)

Considering that Google only has a portion of those indexed, and that at last count there were only 128,324 pages left on that side of your site, that means that at minimum over 21% (and in all likelihood 30% – 40%) of the remaining pages on Mahalo.com are these scraped ones. Is that really what you want on your “Human Powered Search Engine”…?

There are 2 major differences between the original content and your version. 1) The original content cites the sources directly there on the page, whereas Mahalo does not, and 2) Mahalo is using each and every one of these scraped pages to automatically create 2 (and sometimes 3) additional contentless pages under the guise of questions being asked anonymously, questions that no one ever actually asked, but that are there solely for the purpose of bolstering your indexed page count in Google. The first question asked of every drug is What are the side effects of {insert drug}?:

And the second is always Where can I get {insert drug}:

I also saw a “Who makes {insert drug}” question here and there as well. These are empty questions, asked by a bot, that for the most part will never get answered (or even looked at) and were never intended to. Three plus free pages (two of them completely devoid of content) for the price of one scraped page. Jason, seriously, do you really think you are slick doing this?

By the way, a huge number of those pages are less than 100 words in length, yet none of them have the noindex tag. Regardless of of what the length is, however, it’s not what you are claiming Mahalo.com is Jason, it’s more fluff. Wikipedia already gives us those articles. You add nothing to the interwebs by copying them. In all seriousness you should just go through and delete them all.

However, he still kinda kills it by tossing in at the end about how this whole scrutiny on his site is “absurd”, and anyone who calls him on it is being “vicious”:

At the end of the day the absurd microscope we’re being put under by the SEO community is actually making our product better, and yes even improving the SEO of our best quality pages. For that, I thank you guys.

That being said, you really don’t have be so vicious about it. – Jason Calacanis

Jason, there is nothing “absurd” about this shit at all. Ya know, that whole Thou Dost Protest Too Much line? That applies here.

You’re sitting there calling this “vicious” and yet continuing to lie about the situation. You appear to be forgetting that I am not some moron who doesn’t have the facts. You made a statement over on HN and are making a similar statement here trying to implicate that other people put all of those short pages on your site. This isn’t the case, and for ffs, it was explained on Mahalo itself how they got there in the first place. Those pages that I blogged about, and that you are now saying are being noindexed (and which should all be getting removed, not just noindexed) were generated by a bot, one which ran on your site, one that turned people’s searches into static pages, which did so if and only if the page generated clicks, and which then added those pages to your sitemap so Google could more easily find them. This bot was once named “searchclick”, and you actual went out of your way to rename it to “stub” in order to make it sound less conspicuous. Wtf Jason, do you really think there are many things more indicative of awareness of the wrongdoing than going out of your way to actually cover up evidence…?

YOU are the spammer, Jason, you who had people modify whatever original software Mahalo was based on to behave like this. This is not some “consipiricy of the SEO’s” designed to get back at you for anything. You were spamming, acting like a dick when people called you on it, and then on top of that you are getting preferential treatment from Google during this whole thing to boot. Tons of people are getting banned from Google, or AdSense, or both, every day, and they don’t get to talk to a Google rep. They get a form letter and little to no information about what they did wrong (if anything). You get a personal talking to from the head of the Google spam team no less, and you’re response is to whine about how all the “trolls” are being mean to you? Give me a break, Jason.

Matt Cutts himself told you that if you didn’t clean things up that Google might take action. Matt is the head of the Web Spam team. You seriously want us to believe that you thought he was talking about something other than spam when he said that..?

To put things in perspective, not counting the Mahalo Answers side of things (which I haven’t even started to look at) you started out with 12 xml sitemaps each containing up to 50,000 pages in them. The total number of pages was 598,661. After the trimming down, you now have 3 xml sitemaps, with a total of 128,324 pages left. That means that you just whacked 78.6% of the main site, and you still have empty pages in the sitemap. Just looking by hand, it looks like between the remaining empty pages you told me you were going to eventually remove from the sitemap and the fluff that is in there, only about 1 out of 3 of the remaining pages should actually still exist. I pointed one example out on Twitter, but here’s another set so you can again see what I am talking about when I say fluff:

http://www.mahalo.com/jay-z-brooklyn-go-hard
http://www.mahalo.com/jay-z-brooklyn-go-hard-video
http://www.mahalo.com/jay-z-brooklyn-we-go-hard

Same topic with slight rewordings, none with any substantial amounts of content, those could easily be (and should be) a single page. Now, I understand that it takes time to get to all of those pages, but still, the numbers are kind of huge.

If this helps you get a better mental image of why this shit is not an “absurd microscope”, here are some cool infographics for you to look at (these are to scale, by the way, not some made up ratios):

We know those pages didn’t add up to a ton of traffic, Jason, you keep telling us. As I said before it’s not about that, it’s about all the free PageRank they were pulling in. We’ll see where things sit after the cleanup actually gets finished and Google gets around to respidering what is and what isn’t left. My guess? You will be surprised how large an impact this “small portion” of your site was actually having.