It has to be tough policing a program like AdSense. It must be exceptionally difficult during the holiday season, when the payoff to running scams grows so much more. It is so tough, in fact, that this year as the holiday shopping season grows near, with Black Friday just a few short days away, that apparently Google has finally decided to say “fuck it”, make it easier on themselves, just remove the ability for anyone to report any violations of the program whatsoever, and allow the scammers to have a field day in the mean time.

While Google may want to give the impression to their stockholders and the public that they have both the search engine spam and advertising program cheaters fully under control, the truth is that they rely quite a bit on reports from the community and consumers for both spam and AdSense violations. For any spam that they find, Google asks people to submit a Google spam report. At this point they require that someone log in before actually filing the report itself. This makes sense, since it helps prevent people erroneously filing large amount of spam reports against their competitors. For the AdSense violations they supply a separate form that does not require a log in, titled simply Reporting a Violation – AdSense Help. Usually I don’t run into offending sites with AdSense on them that fill me with enough of a sense of civic duty where I feel compelled to actually fill out a report, but I happened to land on one such today that actually tricked me into clicking on an ad in such a way that it really did annoy me. The page I landed on was BigSiteofAmazingFacts How Much Does The Earth Weigh (yes, I was distracted by trivial shit again, don’t judge me), and in the right sidebar there was what appeared to be an embedded Youtube Video from Family Guy:

Still distracted (of course) I clicked Play on the video, only instead of playing it suddenly brought me to a site trying to sell me bras. So, thinking I must have missed the rather large video in the sidebar when I tried to click on it, I hit the back button… and noticed that suddenly the video was gone altogether, and where before I had seen 2 AdSense blocks and a video, now there were 3 AdSense blocks instead:

I hit refresh a few times but the video didn’t return. At that point I realized that it was actually a scam, so I cleared my cookies for that domain, hit refresh again, and viola, the “video” reappeared once again. At this point I was sufficiently irked that I actually decided I was going to report this asshole. It’s bad enough that a site with crap content like this is ranking #1 (the weight of the Earth is increasing each year from salt from the ocean spray? Seriously, wtf?), while people with content that is just fine are getting penalized supposedly from the Panda fallout. To add in that the guy who owns the site is ripping off advertisers as well just makes it so much worse. So, I headed on over to the AdSense Violation report to be a good citizen… and I was greeted by this:

An essentially blank page, with only a header, navigation, and a box asking me to tell AdSense how they can improve. Go figure.

From a financial perspective it does make sense for Google to make reporting AdSense violators more difficult, especially during the holidays. People who run scams like this actually generate Google money through the AdSense program, a program which currently has absolutely no oversight. It is exactly this lack of oversight that means that Google is the only one who knows how much, if any, of the advertising dollars are credited back to the advertisers once these scams are revealed. Hiding the violations report means that much fewer sites will be reported, more scams will be able to run for longer periods of time, and more money will wind up in Google’s pockets.

Is this profit motive really the reason that the report form is missing? If you ask Google I am sure they would say “of course not, we’re Google, you can trust us”. And since everything with Google is proprietary “behind closed doors” trade secrets with them, there is no way to know exactly how many violation reports suddenly went missing that apparently no one has noticed yet. My hunch though is that with something like this, as online shopping hits the holiday rush, the lack of reports that are coming in at the moment is actually too big for them not to have noticed by now, and them not fixing it for this long must be at least in some part intentional on their end.

Update: As Jen from JenSense.com pointed out in the comments, there is another newer page available where you can actually file the report located here. However, I am not sure that makes it any better, and may in fact make it worse. I wound up on the empty page by actually going to Google and searching for [report adsense violation]. The page that Jen provided is in the list, but it is down under the blank page that I found, another unhelpful blank page, and underneath a list of discussion of other people looking for the form. This begs the question… why did Google leave an otherwise empty page behind with just enough text (ie. header and title) and all of the old link juice there to outrank the “real” form? If they redesigned the site, then why not 301 redirect the old form(s) to the new one? It’s not like they don’t know how search engines work, ya know?

[cue elevator music]

Ok, good, you’re back. If you followed some of the aftermath in the comments, on Twitter, and on various media outlets and celebrity blogs around the web (including Gawker and Wil Wheaton), you can tell that Jenny obviously has a large amount of supporters who were less than pleased at Jose Douche Canoe Martinez. The outcry got just loud enough that Brandlink Communications actually started to play the wounded bird card:

Apparently Jose Martinez felt so victimized from the whole experience, he actually decided that he needed to delete his entire Twitter account (or, of course, it could be that he was trying to do the internet equivalent of burning the evidence of his douchiness).

Quick side note: if a PR company’s first instinct when they come under fire is to duck and run, and get defensive, as opposed to owning up, making it right, and the moving on, then odds are that same PR company would not hesitate to throw a client under the bus if they felt it was necessary for their own self preservation. Anyone who is researching this company with the possibility of hiring them should probably keep that in mind.

Either way, I guess Jenny didn’t realize quite how much support she would receive, so she wound up actually asking her followers to put away the pitch forks:

UPDATED: I love you people. Really. Thank you for always having my back and for being so supportive during this weirdness. Jose has apologized, and I’ve been assured by the woman in charge of the company that they are aware and are handling it the best way they know how, so let’s give them some air and let them have the chance to do that. *deep breath* – Jenny Lawson, aka TheBloggess

Ok, so, maybe Jenny is right. Maybe some of her fans did get abusive towards Jose in the process of defending her (which, btw, I did not see myself, but I am guessing not everyone was polite) and it is time for us to let cooler heads prevail. However… I also don’t think this should fall just into internet obscurity, either. People who are looking to hire this PR firm should be able to find out who it is they are dealing with, and the first line of defense when doing research on a company is, of course, Google. Currently when you do a search for [brandlink communications], someone else’s post about Brandlink and TheBloggess is #1 (as a news story though, not as a regular listing), Brandlink Communications themselves show up next (which is actually the natural #1 listing, when no news stories show), and in the natural #10 spot is Jenny herself:

(Click to enlarge)

The news piece is of course only there for a short period of time, as all news pieces should be, and the rankings Jenny’s site has currently are probably also due to what Google refers to as QDF, or Query Deserves Freshness (where a particular search might warrant different results due to topic being “hot” at the moment). However, I think that Jenny’s site should be in the top 10 when searching on that company, even after the buzz dies down… possibly even #1. Therefore, here is what I suggest, if you happen to support Jenny in this issue:

1. Write a post showing your support for Jenny about the way she was treated. Don’t attack or “bully” anyone in the post, because despite them being in the wrong here Brandlink was right, bullying people is still wrong (although calling someone a douche canoe when they actually are one is just being descriptive imo)
2. In that post, link to Jenny’s blog post about the conversation, but use the phrase [brandlink communications] as the anchor text for the actual link.
3. If you link to the post more than once, make sure that link is the first link to the post.

For those of you wondering why, it is because links are still the number one factor Google uses when determining rankings. If you want more information on it, you can Google [santorum] and do some research… just don’t click on the first link.

Do I think this is too harsh? Perhaps if they weren’t a PR company boasting W Hotels and Chase as their clients I might be more inclined to go easy on them. However, even if they weren’t big shots treating those they regard as the “little people” like shit, there is also the fact that this behavior is not new for Jose, and there is evidence of him treating people like this all the way back to early 2006. The fact that the same guy is still VP Media Director 5 1/2 years later, still behaving the same way, makes the promises from the company that is “handling it the best way they know how” somewhat hollow. So yes, with that in mind I think that a response like this is quite fitting. Vote with your links, people, as Google intended you to.

As “pure webspam” has decreased over time, attention has shifted instead to “content farms,” which are sites with shallow or low-quality content. - Matt Cutts

Whoa. This, especially coming from Matt Cutts, is huge. For those who don’t know, “content farms” are organizations that generate websites composed of large amounts of low cost “fluff” or filler content, with little to no regard to quality. The content is generated not based on having information and the desire to share it, but rather in response to queries that might get typed into a search engine, and are built for search spiders rather than human consumption. They include companies like Demand Media, Mahalo, and Associated Content.

Historically speaking, Matt has pretty much refused to come right out and say that these content farms were indeed spam, despite the fact that they clearly violated Google’s quality guidelines:

Doorway pages are typically large sets of poor-quality pages where each page is optimized for a specific keyword or phrase… Google’s aim is to give our users the most valuable and relevant search results. Therefore, we frown on practices that are designed to manipulate search engines and deceive users by directing them to sites other than the ones they selected, and that provide content solely for the benefit of search engines. Google may take action on doorway sites and other sites making use of these deceptive practice, including removing these sites from the Google index. - Google Webmaster Tools Help

Regardless of the very clear wording of their policies, Google has to date not banned any of these content farms for their violations. In fact, quite the opposite – Matt has in the past even defended these sites, and in Mahalo’s case at least given warnings to them which he then allowed them to ignore. He alluded to the fact that one of the algorithm updates from last year, Mayday, was supposed to help filter out “really kind of lower quality” sites, and many people thought he must be talking about content farms back then, but alas that turned out to be a bust. So when he comes right out and says, hey, you’ve waited long enough, now we’re going to target content farms for reals, y’all, then yeah, that’s a Pretty Big Deal.

Now, Richard Rosenblatt, the CEO of Demand Media, may be may be in denial about his company being a content farm, but that definition has existed for quite some time, and regardless of what you call it low quality content built specifically for search engines is in violation of Google’s guidelines. However, he still persists in his belief that as long as you can get some people to call it something else, his “partnership with Google” will keep them protected regardless of what happens:

This is why our partnership with Google makes sense. 1) We help them fill the gaps in their index, where they don’t have quality content. 2) We’re the largest supplier of all video to YouTube, over two billion views and 3) we’re a large AdSense partner. So our relationship is synergistic, and it’s a great partnership. And it’s a partnership that we’re excited to continue to expand. - Richard Rosenblatt, attempting to give Google’s PR team a heart attack

I am guessing that Mr. Rosenblatt missed the section in Matt’s post where he very specifically discussed the fact that no special partnerships would protect the content mills from these changes:

One misconception that we’ve seen in the last few weeks is the idea that Google doesn’t take as strong action on spammy content in our index if those sites are serving Google ads. To be crystal clear:

• Google absolutely takes action on sites that violate our quality guidelines regardless of whether they have ads powered by Google;
• Displaying Google ads does not help a site’s rankings in Google; and
• Buying Google ads does not increase a site’s rankings in Google’s search results.

– Matt Cutts, being crystal clear

Then Friday rolls around, and Matt announces that these changes already happened earlier in the week. If you didn’t notice any changes, then that’s probably because, according to Matt, less than half of a percent of queries would show any perceptible ranking differences. If you didn’t notice any changes in queries involving content farms, well… as near as I can tell that is because there weren’t any. In fact, in his announcement post Matt doesn’t even use the phrase “content farms” at all, and instead only discusses that the net effect of these changes is that in cases where content was scraped, searchers are more likely to see the original content first. He then thanks Jeff Atwood (one of the ones who wrote a story discussing Google’s decline in quality that had a large audience) and Stack Overflow’s team (a site that Jeff co-founded) for their feedback. A few people asked about the omission in the comments, but as of yet anyway Matt has not replied to any of them.

As to the results themselves, for the most part I am seeing what I was seeing before, so that “less than half of a percent” doesn’t surprise me. If you search for [mcdonalds coupons] the #1 site is still a Mahalo page that doesn’t actually have any coupons on it, and very little original content. If you search for [mcdonalds free salad coupons] you get a different Mahalo page that does actually have a picture of a coupon on it (good only in Canada, and expired in July 2010, however), and if you search for [mcdonalds happy meal coupons] the second listing is a Mahalo page, again with no coupons on it. These pages are filled with riveting dialog, such as the section labeled “McDonalds Happy Meal Coupons Coupon Policies,” which states:

The policies for McDonalds Happy Meal coupons may have certain restrictions and these might include not being able to combine discounts or limiting the period of use. Make sure you read and understand the instructions listed on the coupon carefully to ensure that you know when the coupon will become valid and when it will expire as well as what special restrictions apply. Also included in this information will be which product or products the coupon can be used to purchase. Insuring that you understand the coupon policy can help you to avoid any mistakes during the checkout process. - Content Mahalo actually paid for

Seriously?

It’s not just Mahalo, of course… type in [how to reset your blackberry] and you will find ranking just fine a page from eHow that is nothing more than the phrase “hit alt+right-shift+delete” wrapped in light, fluffy filler content. I also still see queries where the duplicate content outranks the original, such as the copy of a Wikipedia page that ranks #1 for [elvett semic]. The changes, whatever they were, truly are barely (if at all) perceptible. The change was so small that one of Matt’s readers asked, “I’m wondering why announce it if you’ve gotten the feedback and the algorithm update would presumably be of such little consequence that no one would likely notice or comment on it unless you told everyone.” Indeed, why make such a big deal out of something when almost no one can tell the difference?

To answer that you need to take a look at exactly what it was that did change. When I search in Google now for questions that were asked on Stack Overflow, at least for the queries I checked, I now see SO ranking instead of sites that scrape their content. This is of course how it should be, and the main concern that the people from that community were bitching giving feedback about to Matt. Stack Overflow is, as I mentioned, the site that was co-founded by Jeff Atwood, who is the author of the much quoted post that generated quite a bit of buzz about Google’s decline in quality. Many of the frequenters of Stack Overflow are also regulars on Hacker News which (not so) coincidentally Matt decided to hold a good portion of the discussion about these changes, both before and after they were implemented. While the HN and SO communities in and of themselves might be tiny compared to the web as a whole, the fact is that their voices do carry within the online community. Start buzz there about Google showing quite a bit of improvement and it has a very good chance of spreading, even if the data set demonstrating that is overall quite small. Add to that the fact that Richard Rosenblatt, CEO of Demand Media knows that the changes aren’t targeted at his company (and when asked if Google had discussed the changes with him, replies “I can’t comment on that.”), and then toss in Jason Calacanis’s ingratiating comments on Matt’s blog post about the changes going live:

It was clear that Mahalo was getting grouped into the “content farm” space… - Jason Calacanis

No kidding? Really? Past tense there, eh Jason?

So Matt loosely ties the concepts of “content farms” and “scrapers” together in a blog post on the official Google Blog, and claims that they are taking action against them. He then announces a change that appears to only affect scraper sites, and furthermore only those scraping a specific dissatisfied community, publicly thanks that community for their help, and then doesn’t mention the phrase “content farm” again. Even though the changes were practically non-existent, there is a good chance that the overall impression from those who don’t look too closely is that action was indeed taken, and that if what were formerly referred to as content farms are still ranking well, then obviously they must be there for a reason.

From a strategic standpoint it’s actually rather clever. If I were Google and I needed to conceal special relationships I had with companies (especially if I was thinking that the FTC might want to get involved in my business) then I too would probably try very hard to sway the public opinion about the labels attached to the sites those companies owned, and shift the focus to something I could fix without caring about the damage, and then crowd source a tech community to help spread the impression that things were better. Most people probably won’t even pay enough attention to notice.

(click to view original)

Personally, I’d like to see Zedo up its standards for the type of ads it will accept. This type of junk shouldn’t be allowed. - Danny Sullivan

He’s right, too, the ad networks should be policing this type of deception, by all means. Matt Cutts, Google’s head of the web spam team, agrees. He tweeted about the story, and also commented his take on the matter in the post itself:

My favorite part of the disclaimer for those type of sites is “This website, and any page on the website, is based loosely off a true story, but has been modified in multiple ways including, but not limited to: the story, the photos, and the comments.”

Oh, so I can trust the website except for the story, photos, and comments? In other words, the entire website?

And if you read the disclaimer carefully, most of these sites promise a “free trial” with $1.95 in shipping, but actually set your card up with a recurring subscription. The “one weird old tip” ad that I clicked from the L.A. Times mentioned this in the fine print: “If you do not cancel within seven (7) days of the date that you enroll in the Program, we will charge the same card you provided at enrollment the non-refundable one-year membership fee of $149.95″. Then they also start charging you $12.95 a month. Grr. – Matt Cutts, on deceptive “flat belly” ads

Grr, indeed.

Danny also mentions in his post about how “The ad, unlike Google’s ads, doesn’t report what ad network is delivering them,” which if they did would be a form of disclosure. And Danny is right… except for one thing. Danny derived the fact that the ad was being served by Zedo by examining the url. However, if you view the source on the LA Times article and go to the spot on the page where the ad is showing, you don’t see the Zedo ad network code. The ad itself is being generated by Javascript that is being pulled from yet another ad network:

The actual ad network that the LA Times has a relationship with, and the ones responsible for what ads show on their site, is Doubleclick. And who owns Doubeclick, you might ask? As most of you probably already know, Google does, since they bought them back in 2007 for $3.1 billion. So obviously not all of the ads Google delivers disclose what network they are from.

It gets better. AdSense, Google’s flagship advertising network, serves what are known as “contextual ads”, where in theory the ad targeting is based on the context of the page contents where the ad blocks are placed. Danny uses AdSense on his site, with one of the blocks being at the very top of the page. Due to the various feeds in the sidebar, the content of the article, and the title, “Acai Berry” is mentioned 8 times on that same page. Therefore it is only natural, of course, that this is what we see when we look at the ads being served on the top:

(click to enlarge)

Now, can you guess where that ad leads? That’s right:

(click to enlarge)

It’s a fake news site identical to the one Danny is discussing, with the same text, layout, and even images embedded in the “story”, with the only variation being that the one Danny landed on is “News 7″, and this one is “News 8″.

What makes this story particularly interesting is that recently Matt Cutts visited Washington D.C., lobbying the FTC about Google’s integrity, trying to convince them that they don’t require government oversight, and how they could be trusted to police themselves. Google also happens to be in a very unique position to help clean up these kinds of abuses. Not only could they pull these ads from their own vast array of properties, and require their third party partners to do the same, but they could also warn publishers who use networks that continue to promote scams that their sites rankings could suffer, in the same way that they have punished websites in the past for what they said was deceptive marketing, in the form of undisclosed paid links. Instead, they themselves appear to be participating in the problem, not the solution.

So, Matt, are you willing to back up your testimony to the FTC about Google’s integrity, and lobby within your own company to help eradicate deceptive marketing from the web? Do you feel that websites that allow deceptive advertising to be shown on their sites should have their trust revoked?

Let’s ignore, for the moment anyways, that for absolutely any other website on the internet with the evidence that I presented the process would most likely entail the webmaster proving their innocence rather than the person reporting the paid link proving that they are guilty, and that they would have to do so after they actually got banned. Before I get started, keep in mind… whatsyourconundrum.com is not Conundrum Wine’s main website. Their actual website is http://conundrumwine.com/. The site that is being linked to from Mahalo is a marketing device, a link or PageRank funnel, something that acts as an intermediary link bait, or simply to increase brand recognition. This is actually a great way to help increase your exposure without having to put gimmicky items on your company’s professional website. That main website is not hosted on Mahalo.com’s servers.

Now, Matt is correct. The whatsyourconundrum.com website is indeed hosted on Mahalo servers. Without even looking at the IP address, we know by looking at the whois record the dns servers are ns1.mahalo.com and ns2.mahalo.com:

The reason, I am guessing, is that apparently whatsyourconundrum.com is powered by a white label version of Mahalo’s Answers. What we also see in the whois record is that the domain is owned by Camus Vinyards, and that their administrative email is domainadmin@caymus.com. Following that hint, we see that the caymus.com domain is owned by Caymus Vinyards:

Caymus.com is in turn being hosted on ewinerysolutions.com servers. Ewinerysolutions.com is being hosted on their own servers, so the dns trail stops there. If we go to the eWinery Solutions website, we can see that what they do is offer marketing services to wineries:

The most profitable and fastest-growing segment of the wine industry today is the consumer-direct channel. eWinery Solutions offers you the freedom to market your wines easily and effectively by creating a one-on-one dialogue with your best customers.

Jot on over to their Portfolio page, and we see that Conundrum Wine is in fact a client of eWinery Solutions:

Now, does eWinery offer their services to Conundrum out of the goodness of their heart? Somehow I doubt it. Did Jason agree to lease and host the white-label version of Mahalo Answers to them because of all of the warm fuzzies he knew it would give him? Of course not. In fact, he even refers to this deal as being Mahalo’s first “client”:

Whether the header link was part of the original deal negotiated to set up whatsyourconundrum.com, or a separate cash transaction, the fact remains that this was not a merit based link. It has it’s roots in a commercial transaction that most likely occurred between someone at Mahalo and the people doing Conundrum’s marketing. What I also can’t say is whether or not the clean sitewide links pointing to whatsyourconundrum.com on Jason’s private blog was part of the deal or not… but there are 5 of those links there as well, under the heading of “Daily Reads”:

Hopefully Matt won’t ask me to get all ninja and obtain a copy of the actual invoice… that might prove a little problematic for me.

Jason sent me an email to correct me on what he was talking about. We replied to each other back and forth a couple times, and a few very interesting things were revealed in that conversation:

On Mon, 2010-01-25 at 14:46 -0800, Jason Calacanis wrote:
sorry, didn’t mean ranking for “video game walkthroughs” literally….
more like VIDEO GAME NAME walkthrough:
http://www.google.com/search?&q=call+of+duty+walkthrough

On Mon, Jan 25, 2010 at 3:38 PM, Michael VanDeMar wrote:
Yeah, realized that and pointed out that I might have misinterpreted in the next comment. I only saw one or two that you were in the top 5 though, not a slew of them. But yeah, better than the generic version.

Thing is that you still have tons of auto or near-auto generated content out there ranked, stuff that is nothing more than noise, including stuff you are boasting about ranking for:

http://www.mahalo.com/need-for-speed-prostreet-walkthrough

I commented about this last year I believe… this is not a legitimate or ethical way to go about what you are doing:

1) Auto-generate tons of pages with no content, with AdSense embedded on them, based solely on search phrases
2) See what winds up ranking
3) Go in and put valid content in afterwards, pretend that all along the content was not only valid, but actually better that what you outranked.

You want a more palatable model? How about this… there are tons of sites out there that are full of great content that no one will ever see, because they’re simply not seo’d (ie. unknown and unlinked). Why don’t you:

1) Find a way to identify the search results that are already full of crap sites
2) Identify quality sites that should be there instead, and
3) Use your pull and popularity to get *those* sites ranked.

If you find a way to do just that and somehow build a successful business model around it then that would be much, much better than what Mahalo is now.

-Michael

On Mon, 2010-01-25 at 15:43 -0800, Jason Calacanis wrote:
the part you’re leaving out is:

a) we used to noindex these and we are going to again
b) if any page gets any traffic we pay someone to build it out–so it’s only short for a couple of days.

if a page is started by a user and is short it a) won’t rank 99% of the time and b) if it happens to we see it in analytics and build it out.

not really a big deal IMO

best j

Let’s take a look a little closer to some of the things that Jason is saying here:

“we used to noindex these and we are going to again”

At one point Mahalo had the noindex tag on the pages that were fully automated, and Jason is freely admitting that they should have the nodindex tag now. These pages have no added value and should not be indexed by Google. Jason must believe this, or he would not say that they were planning to add the noindex them again. If he thought they deserved to be in the index then there is no reason to tell the search engine spiders not to include them. Now, according to Jason’s rebuttal to Aaron, the removal of the nodindex tag was an “accident” that happened in the migration to Mahalo 3.0. For those who do not know, Mahalo 3.0 was released back in November of last year. Mahalo is a template driven site. All it takes to “fix” the tag that was lost “by accident” is a coder opening up the template header and including 1 line of conditional code. In the 3 months prior to our conversation, and in the 3 weeks since, no one has bothered to do so. To this day those pages are still getting added to the index at a very large rate.

How large, you might ask? I have been checking here and there for the past few weeks, and when I looked on any given day Google was reporting from 2,000 to 20,000 new pages on Mahalo.com:

I clicked through quite a few of these pages to see what the content looked like. Almost every singe page I looked at was completely automated content. In fact, when I looked today, the only one I found that wasn’t totally automated was the Mahalo entry on Hootsuite… which happened to have 2 additional manually added sentences in it, content which “enhanced” the scraped content that surrounded it: “HootSuite is the professional Twitter client. With HootSuite, you can manage multiple Twitter profiles, pre-schedule tweets, and measure your success.” The entire “Human Powered” portion of that page is tinier than just one AdSense block located directly under it:

And that AdSense block is one of 4 located on the page. Yeah, let’s hear it for “added value” folks.

As a side note, it looks like Jason is violating Google AdSense TOS by placing more than 3 AdSense blocks on each page. Not entirely sure why they are letting him do that, since supposedly that lowers the overall value each advertiser gets… but I digress.

“if any page gets any traffic we pay someone to build it out”

Here Jason is clearly admitting that he thinks it’s fine to rank your scraped content first, and then add quality if and only if it gets traffic (and remember, 1-2 sentences of “quality” is fine). Essentially he’s saying, go ahead and spam, but if it looks like it might get enough traffic someone will actually notice, add content.

I wrote back:

On Mon, Jan 25, 2010 at 4:51 PM, Michael VanDeMar wrote:
So, you’re not denying that is indeed your model here (rank first, quality later), you’re just saying that you think that’s it’s fine to do it that way, since you pay people to build it out within a couple of days after you start to get traffic?

Your assertion that a page on your devoid of content won’t rank is completely untrue, by the way. Generally people with quality content wind up ranking because other people link to them. That’s what the whole basis of Google’s base algorithm is built upon. Your site, however, gets pages ranked solely by virtue of other internal pages linking to them. You have so much link juice, due mostly to controversy and press, ranking power that you spread around your site from one page to another, that you can rank relatively competitive phrases with no effort at all. That does not somehow make the page quality due to some sort of mystical link juice feedback. Quality attracts link juice. Link juice does not impart quality.

Take a look at these pages, all which rank with pretty much nothing but links from within Mahalo itself:

http://www.google.com/search?q=valentine%27s+day+cupcakes
#1: http://www.mahalo.com/valentines-day-cupcakes
1 external link, from a scraper: http://search.yahoo.com/search?p=link%3Ahttp%3A%2F%2Fwww.mahalo.com%2Fvalentines-day-cupcakes+-site%3Amahalo.com

http://www.google.com/search?hl=en&safe=off&num=10&q=halo+3+walkthrough&btnG=Search
#5: http://www.mahalo.com/halo-3-walkthrough
Again, only linked externally from another scraper: http://search.yahoo.com/search?p=link%3Ahttp%3A%2F%2Fwww.mahalo.com%2Fhalo-3-walkthrough+-site%3Amahalo.com

http://www.google.com/search?q=how+to+invest+online
#8: http://www.mahalo.com/how-to-invest-online
Zero links from external sites, even from scrapers: http://search.yahoo.com/search?p=link%3Ahttp%3A%2F%2Fwww.mahalo.com%2Fhow-to-invest-online+-site%3Amahalo.com

Google describes it’s ranking algorithm as something that leverages the democratic nature of the web. No one, however, is voting for your pages. Why should they rank? Simply because you paid someone to flesh them out?

-Michael

PS. Do you mind if I blog this? Or are any of your answers being said with an expectation of privacy?

On Mon, 2010-01-25 at 17:00 -0800, Jason Calacanis wrote:
Actually, most of our pages come in VERY LARGE now because we don’t allow folks to just make them on the live site any more. They have to go through Mahalo Tasks now. in the old days it was a free for all like Wikipedia/Squidoo…. we didn’t like the results.

Very few of our pages start short. So, it’s not really a strategy to put up short pages and wait… our strategy is to get a TON of people to make pages in Mahalo Tasks.

Think about it: which is a better way to build a business… make a bunch of stubs or make a bunch of high-quality pages? The later is better and it’s really not very expensive. Especially now that we have revenue sharing with our page managers.

The three pages you’re talking about all have a LARGE amount of original content, and I would say they are B+ to A+ content, no? I think google and yahoo take into account a large amount of content plus the amount of time a user sits on a page. At least i’ve been told that they look at time spent on page. as you can imagine… the time spent on a recipe, walkthrough or howto article is VERY LONG.

thanks for the feedback!

best j

Again, let’s look at what he’s saying here:

“we don’t allow folks to just make [pages] on the live site any more. They have to go through Mahalo Tasks now.”

That’s right, Jason would like us to believe that the Mahalo pages, like the one on 13 year olds and rape, all now pass an editorial review before going live. Even if that were true (which it isn’t, btw) I don’t think that I would be boasting that, based on the pages that I looked at.

in the old days it was a free for all like Wikipedia/Squidoo…. we didn’t like the results.

Now Jason Calacanis is saying that the 100% user contributed, zero ads Wikipedia is nothing more than a “free for all”, and that his 99% auto-generated content content laden with AdSense is therefore better. Gotcha.

“Very few of our pages start short.”

Either Jason thinks he’s talking about a different site, or he has absolutely no concept of what the phrase “very few” actually means. Currently when I look, Google tells me that Mahalo has 356,000 pages indexed:

As I mentioned earlier, in contrast to “very few” of these pages being “short”, the vast majority of them are nothing more than scraped content. From what I can tell it looks like less than 4% of the Mahalo.com pages getting indexed on a daily basis actually involve humans adding unique content in to them. The rest of the pages are being auto-generated by a Mahalo bot aptly named “searchclick”. If you go to any of these non-content pages and click on the “View Page History” link in the bottom of the right column you can see exactly who it is that generates the majority of this “Human Powered Search” website:

“searchclick” is of course not an actual user. It’s the name of the robot that Mahalo uses to generate all of these pages on it’s site. According to the Mahalo website,

Searchclick is not really a user at all. The name “searchclick” is used to represent a individual “search click” by a Mahalo user who has searched a particular term and Mahalo made a “created by searchclick” meaning the page had been searched for the first time.

These pages on Mahalo that are getting indexed by the thousands are nothing more than searches that users performed that no one thought valuable enough to create a topic page for in the first place. The content used to populate these pages is nothing more than scraped versions of other websites search pages, such as Youtube, Flickr, and Google itself.

Google is very clear on it’s viewpoint of indexing search pages. Matt Cutts, head of Google’s Webspam team, wrote about the subject back in 2007:

But just to close the loop on the original question on that thread and clarify that Google reserves the right to reduce the impact of search results and proxied copies of web sites on users, Vanessa also had someone add a line to the quality guidelines page. The new webmaster guideline that you’ll see on that page says “Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines.” – Matt Cutts

There really is not a whole lot of room for ambiguity in what Matt is saying.

As it turns out, Mahalo can not block these pages using robots.txt. They go out of their way to make these search generated pages blend in with every other page on their site. Not only are they all located in the same root directory, there is nothing at all that allows the fact that they are auto-generated to be detected in any sort of machine readable way. Is this by accident? From the way Jason talks, you would think so. However…

The mere creation of pages is not enough to get them discovered and indexed in the search engines. The search engine spiders must have some means of locating these pages in order to know that they exist. Most sites simply rely on links from other pages on their sites, or links from other people’s sites, for their content to get noticed. There are alternate means though. For instance, Google allows webmasters to set up special sitemaps, meant for search engine spiders alone, in order to insure that sets of pages that the webmaster considers important do in fact get found. These spider-only sitemaps are in XML format, and are not meant for human visitors to use. Does Mahalo use XML sitemaps?

Yep, they sure do. In fact, they use a dynamically generated sitemap, one that is generated on the fly with a url parameter, “p=”, used to distinguish which page of the sitemap you want to view. If we actually open up Mahalo’s sitemap, what do we see?

That’s right… Mahalo’s XML sitemap is packed with tons of spammy auto-generated pages for Google find. Not only is Mahalo not blocking these pages from getting indexed, they are actually going out of their way to make sure Googlebot finds them.

While Jason’s assertion that these pages don’t directly generate a ton of traffic for him may in fact be true, they do play a very important role in his overall ranking scheme. With nothing but the power of the domain they reside on many of these very little competition, zero traffic phrases will actually result in rankings. On occasion you will find spammers scraping these rankings from Google, which then result in a very low pagerank link back to Mahalo.com. Individually each of these links adds almost no value. However, as I mentioned earlier, Mahalo has hundreds of thousands of these pages indexed. In conjunction with link juice from 1 or 2 links from his own blog often times Jason can now get some moderately competitive phrases ranked with no effort, or votes from the rest of the world, at all. For instance, currently at the top of many pages on Mahalo you will see a link to the page for Robert Kissel, who happens to be in the news currently. Search on Google for him and you can see that Mahalo page is currently in the top 10. If we look at the links pointed to this page we see that aside from interior pages on Mahalo.com, exactly 4 sites link to it, 3 of which are from scraping Google’s results:

Even if we are to believe that the Jason himself believes the naive assertion that the sheer amount of content or time a visitor spends on a page are major ranking factors, we all know that those alone will not get you ranked, even if your content truly is quality. As to Jason’s statement that the pages on Mahalo are “quality” and deserve to rank? In most of the cases that I saw, and I am talking about the “human powered” ones now, they simply aren’t. Take for instance Mahalo’s top 10 ranking for [need for speed prostreet walkthrough], if you look at the page there isn’t even a walkthrough on it:

Aside from that tiny little blurb and links scraped from the search engines, the only thing on that page is an embeded video of a walkthrough that was made by someone other than a Mahalo user… and that video doesn’t even exist anymore. If Mahalo used content that they generated and hosted themselves that would never be an issue.

At this point in the game of course, simply noindexing the pages in and of itself is not really a solution. Unless Mahalo moves the search auto-generated content into it’s own subdirectory so it can be blocked by robots.txt, noindexes and nofollows the existing pages (to prevent grabbing unwarranted juice from serps scrapers), and removes them from their sitemap, then they are in clear violation of Google’s Webmaster Guidelines. Why Google won’t actually take action on them is anyone’s guess.

I mean, seriously… wtf??