Smackdown! » search engines

Google Says “Fuck It” For The Christmas Season, Removes The Ability To Report AdSense Violations

Michael VanDeMar — Tue, 22 Nov 2011 20:57:49 +0000

It has to be tough policing a program like AdSense. It must be exceptionally difficult during the holiday season, when the payoff to running scams grows so much more. It is so tough, in fact, that this year as the holiday shopping season grows near, with Black Friday just a few short days away, that apparently Google has finally decided to say “fuck it”, make it easier on themselves, just remove the ability for anyone to report any violations of the program whatsoever, and allow the scammers to have a field day in the mean time.

While Google may want to give the impression to their stockholders and the public that they have both the search engine spam and advertising program cheaters fully under control, the truth is that they rely quite a bit on reports from the community and consumers for both spam and AdSense violations. For any spam that they find, Google asks people to submit a Google spam report. At this point they require that someone log in before actually filing the report itself. This makes sense, since it helps prevent people erroneously filing large amount of spam reports against their competitors. For the AdSense violations they supply a separate form that does not require a log in, titled simply Reporting a Violation – AdSense Help. Usually I don’t run into offending sites with AdSense on them that fill me with enough of a sense of civic duty where I feel compelled to actually fill out a report, but I happened to land on one such today that actually tricked me into clicking on an ad in such a way that it really did annoy me. The page I landed on was BigSiteofAmazingFacts How Much Does The Earth Weigh (yes, I was distracted by trivial shit again, don’t judge me), and in the right sidebar there was what appeared to be an embedded Youtube Video from Family Guy:

Still distracted (of course) I clicked Play on the video, only instead of playing it suddenly brought me to a site trying to sell me bras. So, thinking I must have missed the rather large video in the sidebar when I tried to click on it, I hit the back button… and noticed that suddenly the video was gone altogether, and where before I had seen 2 AdSense blocks and a video, now there were 3 AdSense blocks instead:

I hit refresh a few times but the video didn’t return. At that point I realized that it was actually a scam, so I cleared my cookies for that domain, hit refresh again, and viola, the “video” reappeared once again. At this point I was sufficiently irked that I actually decided I was going to report this asshole. It’s bad enough that a site with crap content like this is ranking #1 (the weight of the Earth is increasing each year from salt from the ocean spray? Seriously, wtf?), while people with content that is just fine are getting penalized supposedly from the Panda fallout. To add in that the guy who owns the site is ripping off advertisers as well just makes it so much worse. So, I headed on over to the AdSense Violation report to be a good citizen… and I was greeted by this:

An essentially blank page, with only a header, navigation, and a box asking me to tell AdSense how they can improve. Go figure.

From a financial perspective it does make sense for Google to make reporting AdSense violators more difficult, especially during the holidays. People who run scams like this actually generate Google money through the AdSense program, a program which currently has absolutely no oversight. It is exactly this lack of oversight that means that Google is the only one who knows how much, if any, of the advertising dollars are credited back to the advertisers once these scams are revealed. Hiding the violations report means that much fewer sites will be reported, more scams will be able to run for longer periods of time, and more money will wind up in Google’s pockets.

Is this profit motive really the reason that the report form is missing? If you ask Google I am sure they would say “of course not, we’re Google, you can trust us”. And since everything with Google is proprietary “behind closed doors” trade secrets with them, there is no way to know exactly how many violation reports suddenly went missing that apparently no one has noticed yet. My hunch though is that with something like this, as online shopping hits the holiday rush, the lack of reports that are coming in at the moment is actually too big for them not to have noticed by now, and them not fixing it for this long must be at least in some part intentional on their end.

Update: As Jen from JenSense.com pointed out in the comments, there is another newer page available where you can actually file the report located here. However, I am not sure that makes it any better, and may in fact make it worse. I wound up on the empty page by actually going to Google and searching for [report adsense violation]. The page that Jen provided is in the list, but it is down under the blank page that I found, another unhelpful blank page, and underneath a list of discussion of other people looking for the form. This begs the question… why did Google leave an otherwise empty page behind with just enough text (ie. header and title) and all of the old link juice there to outrank the “real” form? If they redesigned the site, then why not 301 redirect the old form(s) to the new one? It’s not like they don’t know how search engines work, ya know?

How Matt Cutts Leveraged The Stack Overflow And Hacker News Communities In Redefining The Phrase “Content Farms”

Michael VanDeMar — Mon, 31 Jan 2011 21:52:15 +0000

A little over a week ago, on the Friday before last, Matt Cutts, the head of Google’s Web Spam Team, wrote a post on the Official Google Blog titled “Google search and search engine spam”. This post, and the upcoming changes it discussed, were most likely in response to a growing trend of dissatisfaction with Google’s results that have been cropping up around the blogosphere. In the post Matt talks about how Google feels that things are in fact not as bad as people are saying, and that “Google’s search quality is better than it has ever been in terms of relevance, freshness and comprehensiveness.” He does say that recently, due to increase in both “size and freshness” that of course some spam did get indexed, and also states that as the old, tired, run of the mill spam decreased in Google’s index that Google will now be shifting it’s focus on to content that just sucks:

As “pure webspam” has decreased over time, attention has shifted instead to “content farms,” which are sites with shallow or low-quality content. - Matt Cutts

Whoa. This, especially coming from Matt Cutts, is huge. For those who don’t know, “content farms” are organizations that generate websites composed of large amounts of low cost “fluff” or filler content, with little to no regard to quality. The content is generated not based on having information and the desire to share it, but rather in response to queries that might get typed into a search engine, and are built for search spiders rather than human consumption. They include companies like Demand Media, Mahalo, and Associated Content.

Historically speaking, Matt has pretty much refused to come right out and say that these content farms were indeed spam, despite the fact that they clearly violated Google’s quality guidelines:

Doorway pages are typically large sets of poor-quality pages where each page is optimized for a specific keyword or phrase… Google’s aim is to give our users the most valuable and relevant search results. Therefore, we frown on practices that are designed to manipulate search engines and deceive users by directing them to sites other than the ones they selected, and that provide content solely for the benefit of search engines. Google may take action on doorway sites and other sites making use of these deceptive practice, including removing these sites from the Google index. - Google Webmaster Tools Help

Regardless of the very clear wording of their policies, Google has to date not banned any of these content farms for their violations. In fact, quite the opposite – Matt has in the past even defended these sites, and in Mahalo’s case at least given warnings to them which he then allowed them to ignore. He alluded to the fact that one of the algorithm updates from last year, Mayday, was supposed to help filter out “really kind of lower quality” sites, and many people thought he must be talking about content farms back then, but alas that turned out to be a bust. So when he comes right out and says, hey, you’ve waited long enough, now we’re going to target content farms for reals, y’all, then yeah, that’s a Pretty Big Deal.

Now, Richard Rosenblatt, the CEO of Demand Media, may be may be in denial about his company being a content farm, but that definition has existed for quite some time, and regardless of what you call it low quality content built specifically for search engines is in violation of Google’s guidelines. However, he still persists in his belief that as long as you can get some people to call it something else, his “partnership with Google” will keep them protected regardless of what happens:

This is why our partnership with Google makes sense. 1) We help them fill the gaps in their index, where they don’t have quality content. 2) We’re the largest supplier of all video to YouTube, over two billion views and 3) we’re a large AdSense partner. So our relationship is synergistic, and it’s a great partnership. And it’s a partnership that we’re excited to continue to expand. - Richard Rosenblatt, attempting to give Google’s PR team a heart attack

I am guessing that Mr. Rosenblatt missed the section in Matt’s post where he very specifically discussed the fact that no special partnerships would protect the content mills from these changes:

One misconception that we’ve seen in the last few weeks is the idea that Google doesn’t take as strong action on spammy content in our index if those sites are serving Google ads. To be crystal clear:

Google absolutely takes action on sites that violate our quality guidelines regardless of whether they have ads powered by Google;

Displaying Google ads does not help a site’s rankings in Google; and

Buying Google ads does not increase a site’s rankings in Google’s search results.

– Matt Cutts, being crystal clear

Then Friday rolls around, and Matt announces that these changes already happened earlier in the week. If you didn’t notice any changes, then that’s probably because, according to Matt, less than half of a percent of queries would show any perceptible ranking differences. If you didn’t notice any changes in queries involving content farms, well… as near as I can tell that is because there weren’t any. In fact, in his announcement post Matt doesn’t even use the phrase “content farms” at all, and instead only discusses that the net effect of these changes is that in cases where content was scraped, searchers are more likely to see the original content first. He then thanks Jeff Atwood (one of the ones who wrote a story discussing Google’s decline in quality that had a large audience) and Stack Overflow’s team (a site that Jeff co-founded) for their feedback. A few people asked about the omission in the comments, but as of yet anyway Matt has not replied to any of them.

As to the results themselves, for the most part I am seeing what I was seeing before, so that “less than half of a percent” doesn’t surprise me. If you search for [mcdonalds coupons] the #1 site is still a Mahalo page that doesn’t actually have any coupons on it, and very little original content. If you search for [mcdonalds free salad coupons] you get a different Mahalo page that does actually have a picture of a coupon on it (good only in Canada, and expired in July 2010, however), and if you search for [mcdonalds happy meal coupons] the second listing is a Mahalo page, again with no coupons on it. These pages are filled with riveting dialog, such as the section labeled “McDonalds Happy Meal Coupons Coupon Policies,” which states:

The policies for McDonalds Happy Meal coupons may have certain restrictions and these might include not being able to combine discounts or limiting the period of use. Make sure you read and understand the instructions listed on the coupon carefully to ensure that you know when the coupon will become valid and when it will expire as well as what special restrictions apply. Also included in this information will be which product or products the coupon can be used to purchase. Insuring that you understand the coupon policy can help you to avoid any mistakes during the checkout process. - Content Mahalo actually paid for

Seriously?

It’s not just Mahalo, of course… type in [how to reset your blackberry] and you will find ranking just fine a page from eHow that is nothing more than the phrase “hit alt+right-shift+delete” wrapped in light, fluffy filler content. I also still see queries where the duplicate content outranks the original, such as the copy of a Wikipedia page that ranks #1 for [elvett semic]. The changes, whatever they were, truly are barely (if at all) perceptible. The change was so small that one of Matt’s readers asked, “I’m wondering why announce it if you’ve gotten the feedback and the algorithm update would presumably be of such little consequence that no one would likely notice or comment on it unless you told everyone.” Indeed, why make such a big deal out of something when almost no one can tell the difference?

To answer that you need to take a look at exactly what it was that did change. When I search in Google now for questions that were asked on Stack Overflow, at least for the queries I checked, I now see SO ranking instead of sites that scrape their content. This is of course how it should be, and the main concern that the people from that community were ~~bitching~~ giving feedback about to Matt. Stack Overflow is, as I mentioned, the site that was co-founded by Jeff Atwood, who is the author of the much quoted post that generated quite a bit of buzz about Google’s decline in quality. Many of the frequenters of Stack Overflow are also regulars on Hacker News which (not so) coincidentally Matt decided to hold a good portion of the discussion about these changes, both before and after they were implemented. While the HN and SO communities in and of themselves might be tiny compared to the web as a whole, the fact is that their voices do carry within the online community. Start buzz there about Google showing quite a bit of improvement and it has a very good chance of spreading, even if the data set demonstrating that is overall quite small. Add to that the fact that Richard Rosenblatt, CEO of Demand Media knows that the changes aren’t targeted at his company (and when asked if Google had discussed the changes with him, replies “I can’t comment on that.”), and then toss in Jason Calacanis’s ingratiating comments on Matt’s blog post about the changes going live:

It was clear that Mahalo was getting grouped into the “content farm” space… - Jason Calacanis

No kidding? Really? Past tense there, eh Jason?

So Matt loosely ties the concepts of “content farms” and “scrapers” together in a blog post on the official Google Blog, and claims that they are taking action against them. He then announces a change that appears to only affect scraper sites, and furthermore only those scraping a specific dissatisfied community, publicly thanks that community for their help, and then doesn’t mention the phrase “content farm” again. Even though the changes were practically non-existent, there is a good chance that the overall impression from those who don’t look too closely is that action was indeed taken, and that if what were formerly referred to as content farms are still ranking well, then obviously they must be there for a reason.

From a strategic standpoint it’s actually rather clever. If I were Google and I needed to conceal special relationships I had with companies (especially if I was thinking that the FTC might want to get involved in my business) then I too would probably try very hard to sway the public opinion about the labels attached to the sites those companies owned, and shift the focus to something I could fix without caring about the damage, and then crowd source a tech community to help spread the impression that things were better. Most people probably won’t even pay enough attention to notice.

(click to view original)

Matt Cutts Criticizes Deceptive Ads, Doesn’t Realize Google Is The One Serving Them

Michael VanDeMar — Sun, 30 Jan 2011 21:00:56 +0000

Yesterday over on Daggle.com Danny Sullivan published a post titled, Of Misleading Acai Berry Ads & Fake Editorial Sites. In the article Danny discuses a rising trend of deceptive marketing practices involving fake news sites, the way they rip people off with products they are selling, and the fact that authority sites such as the LA Times are the ones carrying these ads, lending them some credibility in the public eye. Danny states in the post that the ads showing are being served by Zedo, and that he wishes the ad network should raise it’s standards and not allow such blatantly misleading advertising:

Personally, I’d like to see Zedo up its standards for the type of ads it will accept. This type of junk shouldn’t be allowed. - Danny Sullivan

He’s right, too, the ad networks should be policing this type of deception, by all means. Matt Cutts, Google’s head of the web spam team, agrees. He tweeted about the story, and also commented his take on the matter in the post itself:

My favorite part of the disclaimer for those type of sites is “This website, and any page on the website, is based loosely off a true story, but has been modified in multiple ways including, but not limited to: the story, the photos, and the comments.”

Oh, so I can trust the website except for the story, photos, and comments? In other words, the entire website?

And if you read the disclaimer carefully, most of these sites promise a “free trial” with $1.95 in shipping, but actually set your card up with a recurring subscription. The “one weird old tip” ad that I clicked from the L.A. Times mentioned this in the fine print: “If you do not cancel within seven (7) days of the date that you enroll in the Program, we will charge the same card you provided at enrollment the non-refundable one-year membership fee of $149.95″. Then they also start charging you $12.95 a month. Grr. – Matt Cutts, on deceptive “flat belly” ads

Grr, indeed.

Danny also mentions in his post about how “The ad, unlike Google’s ads, doesn’t report what ad network is delivering them,” which if they did would be a form of disclosure. And Danny is right… except for one thing. Danny derived the fact that the ad was being served by Zedo by examining the url. However, if you view the source on the LA Times article and go to the spot on the page where the ad is showing, you don’t see the Zedo ad network code. The ad itself is being generated by Javascript that is being pulled from yet another ad network:

The actual ad network that the LA Times has a relationship with, and the ones responsible for what ads show on their site, is Doubleclick. And who owns Doubeclick, you might ask? As most of you probably already know, Google does, since they bought them back in 2007 for $3.1 billion. So obviously not all of the ads Google delivers disclose what network they are from.

It gets better. AdSense, Google’s flagship advertising network, serves what are known as “contextual ads”, where in theory the ad targeting is based on the context of the page contents where the ad blocks are placed. Danny uses AdSense on his site, with one of the blocks being at the very top of the page. Due to the various feeds in the sidebar, the content of the article, and the title, “Acai Berry” is mentioned 8 times on that same page. Therefore it is only natural, of course, that this is what we see when we look at the ads being served on the top:

(click to enlarge)

Now, can you guess where that ad leads? That’s right:

(click to enlarge)

It’s a fake news site identical to the one Danny is discussing, with the same text, layout, and even images embedded in the “story”, with the only variation being that the one Danny landed on is “News 7″, and this one is “News 8″.

What makes this story particularly interesting is that recently Matt Cutts visited Washington D.C., lobbying the FTC about Google’s integrity, trying to convince them that they don’t require government oversight, and how they could be trusted to police themselves. Google also happens to be in a very unique position to help clean up these kinds of abuses. Not only could they pull these ads from their own vast array of properties, and require their third party partners to do the same, but they could also warn publishers who use networks that continue to promote scams that their sites rankings could suffer, in the same way that they have punished websites in the past for what they said was deceptive marketing, in the form of undisclosed paid links. Instead, they themselves appear to be participating in the problem, not the solution.

So, Matt, are you willing to back up your testimony to the FTC about Google’s integrity, and lobby within your own company to help eradicate deceptive marketing from the web? Do you feel that websites that allow deceptive advertising to be shown on their sites should have their trust revoked?

Breaking News: Google Borks the Earth

Michael VanDeMar — Mon, 23 Aug 2010 21:22:53 +0000

Want to explore the entire planet from your computer? Normally all anyone wanting to do so would have to do would be to trot on over to Google Earth, download and install their application, and off globe trotting they could go. Today, unfortunately, those who do not already have the program installed are apparently out of luck. It looks like today one of the brighter Google engineers working for one of the world’s leading tech companies has somehow broken not just one of the download links for the application, but all of them.

The first place many people would find one of the download links is right in the Google serps, once under the Google Earth sitelinks and once as it’s own listing:

That particular download link, earth.google.com/download-earth.html, is being redirected to what I am guessing is an agreement page, http://www.google.com/earth/download/ge/agree.html. This, however, returns a 404:

The second place people could normally download Google Earth from would be to go to the Google Earth homepage, which was previously located at earth.google.com, but is now being redirected to http://www.google.com/earth/index.html. There you can find 2 links, one in the left navigation and one as a large blue button with the text “Download Google Earth 5″:

As inviting as that button is, however, it is simply teasing you. Both the link and the button trigger a Javascript function named earth.downloadEarth(). Normally downloading the entire planet would be a huge power trip… today however you get from clicking the button is “Server not found”:

It looks like the reason for this one not working is because someone got sloppy when changing the links from earth.google.com to www.google.com, and simply combined the two into http://earth.googlewww.google.com/intl/en/download-earth.html, although that particular page doesn’t exist on either domain so obviously they messed up more than once. Also, what is even odder, is that the Google Earth packages are also missing from the Ubuntu download repositories:

To have Google Earth not be installable from anywhere seems almost as if there is something deliberate going on. Is Google going to phase out one of it’s cooler applications? Or is something new coming down the pipes from them that will replace it? Only time will tell.

Thanks to Donna Fontenot for discovering this today!

zOMG! Jason Calacanis Lied Again?? Shocker!

Michael VanDeMar — Mon, 21 Jun 2010 12:45:58 +0000

Last Thursday, in response to Matt Cutts stating that he needed more than “arbitrary inurl searches” to sway him (which was in turn in response to a Hacker News submission about Mahalo and the plethora of keyword rich domains they were apparently building out) I wrote a post explaining in some detail how the latest Mahalo spam is in fact spam. I demonstrated in the post how Jason had developed a linkfarm which was being used as a link source back to Mahalo.com. It wasn’t just that the individual sites were all linking back to the mother site, which would in fact be normal, but also that the pages were linking back to specific pages within the main site, pages that in many cases had few, if any, links going to them aside from the ones from this linkfarm.

Each time it happens Matt’s defense of Mahalo spamming Google just gets more perplexing. In this latest round he started by saying that his job was not to have knee jerk reactions, as if Mahalo hadn’t already established a pattern of spamming over a long period of time, and that Matt is pretending he hadn’t already had a talk with Jason and told him that if he didn’t raise the bar with his site that Google would take action on Mahalo. From there it got even weirder – Matt looked at the linkfarm and basically told me that a) he didn’t care as long as it wasn’t passing link juice, and b) he’s the only one who could tell if that was the case.

I could have sworn that it was if you were caught trying to spam you were penalized, and you couldn’t get the penalty removed unless you promised not to do it again. Now, where did I get such a crazy and wild idea? Oh yeah, I remember now… it was from Matt Cutts:

Now we come to the heart of things: what goes into a reinclusion request. Fundamentally, Google wants to know two things: 1) that any spam on the site is gone or fixed, and 2) that it’s not going to happen again. – Matt Cutts on the bare essentials of a reconsideration request

The reasons Matt gives out for defending Mahalo seem to be getting more and more creative (even if not more believable). Jason’s, on the other hand, are the same old song and dance he has been spouting since I first called him on his bs and demonstrated that the vast majority of his site was nothing more than empty, auto-generated pages. On Thursday’s post, before he started to lose it with his “fuck you losers, I’m rich” tirade, Jason made this statement:

We have humans write pages of at least 300 words. We don’t index 99.99% of pages with < 300 (it would have to be something unique), and we police the system to get short pages up to 300 words within 30 days. - Jason Calacanis, 4 days ago

Orly? Let’s take a look at those claims, shall we?

The Mahalo coupon pages are about the crappiest pages I have found on the site. When I was doing my initial investigation I stumbled across quite a few of them. My guess is that [{brand} coupon] generates AdSense blocks with a decent eCPM since they are, after all, “targeted” pages. None of the Mahalo “coupon” pages actually have any coupons, which of course means that the end user is much more likely to click on one of the ads when they land there, and more required clicks does means a poorer user experience. What content these pages do have is fluff text that gives ample opportunity for Mahalo to link back to itself, and have spammy signals that are easy to spot like when there are near-identical versions of the same topic page, usually by doing one page for “coupons” and another for “printable coupons” (and no, there is nothing to print out on those pages either). Therefore i picked those as where I would look first to point out, yet again, how Jason was simply pulling these claims out of his ass with no supporting truths behind them.

Digging back into my old data, from March 13th, I was able to determine that from the day the site started adding content up until that point in time Mahalo had amassed 2,655 coupon based pages. When I re-scanned and looked this time I found that there was now 16,601 of these pages. That is a huge increase for 3 months, and a ton of content to create uniquely, even if you ditch quality altogether. Mahalo currently only has a grand total of 90,494 of actual pages on that side of things, so that means 18% of the site is made up of “coupon” pages – and by that I mean coupon pages that don’t actually have any coupons on them.

What’s more, it actually looks like there is a chance that 9,932 of those pages were added last week, over a 3 day period. How the hell do you get writers to create 9,932 pages of even crappy content, all about coupons, in only 3 days?

As I started looking into it I suddenly understood… they didn’t just ditch the quality to create those pages, they went ahead and ditched the content, yet again. I checked over 30 pages, and time after time I found what I found was auto-generated pages that were nothing but ads, affiliate links, and scraper feeds.

http://www.mahalo.com/1800pools-coupons:

(click to view full page screenshot)

http://www.mahalo.com/tigerdirect-coupons:

(click to view full page screenshot)

http://www.mahalo.com/topnotchcare-com-coupons:

(click to view full page screenshot)

Most of the pages I checked had the affiliate links provided by Savings.com, and most linked to the same two questions pages: one discussing the Outback coupons page, and one discussing “grocery coupons”… and in every case neither question had anything to do with what the actual “coupon” page was supposedly about:

For the pages that did not have Savings.com affiliate feeds on them it was because they were using as keywords the names of sites that wouldn’t actually be Savings.com publishers, like GBB.org and RLS Forum. It looks like Jason somehow got his hands on a list of sites that for some reason or another looked like they might have offered some sort of coupon. These were then dumped into the database in the form of pages, and were then checked to see if they matched up with the Savings.com feed. If they did, great, if not that’s ok too, they still had AdSense on them – despite the fact that putting AdSense on pages without actual content is a direct violation of Google AdSense policies:

That’s ok though, I am sure Jason doesn’t care that he is risking the bulk of the site’s revenue stream by violating the terms of the program, since it looks like the AdSense team is giving him just as much of a pass as the spam team is.

In addition to the pages simply being devoid of content, Jason also uses the tactic of creating near-duplicate versions of some of these pages in order to get the most out of the long-tail phrase variations:

http://www.mahalo.com/1and1-coupons
http://www.mahalo.com/1and1-internet-coupons
http://www.mahalo.com/1and1-web-hosting-coupons
http://www.mahalo.com/1and1affiliate-com-coupons

Let’s look at Jason’s statements again…

We have humans write pages

Well, no. You have humans write some pages, but an assload are still auto-generated. In addition to the ones shown here, Google also says that you still have 13,200 pages that you scraped from Wikipedia in their index:

Adding the above auto-generated pages in with the Wikipedia ones, that means that at this point an estimated 33% of the Mahalo content pages are scraped or auto-generated, and that’s just the stuff that’s easy to find. Yay footprints.

of at least 300 words

Again, no, even on the human generated pages that is not always true. Take a look, for instance, at the 1and1 page on Mahalo.com that all 4 of the above coupons reference:

Including words of 3 letters and less that page still only has 212 words of human generated content on it. I also pointed out last week that some of the Wikipedia scraped pages remained thin, such as the one on “The Alice B. Toklas Cookbook”, which has only 261 words on it.

We don’t index 99.99% of pages with < 300 [words]

Bullshit. Not one single one of the pages I examined had a “noindex” tag on it, or was blocked by robots.txt. In fact, just the opposite – every single one of them was pushed to Mahalo’s sitemap, to make it easier for Google to find (and index) them.

we police the system to get short pages up to 300 words within 30 days

Again, bullshit. The 1and1 page has been that way since at least March 11th:

And the Alice B. Toklas one since March 12th:

So Jason, please, enough with the bs. Quit claiming stuff that simply is not true, especially when it’s so damn easy to disprove what you say. I still have no idea why it is that Matt Cutts is choosing to ignore your spam, but to the rest of us it’s as plain as day. And no, Jason… going in now and trying to clean it up in no way changes the fact that you spammed in the first place.

Was The Google Mayday Update A Complete Failure Then?

Michael VanDeMar — Fri, 11 Jun 2010 15:52:35 +0000

Earlier this week at SMX Advanced Seattle, during the You&A With Matt Cutts, the topic of the latest Google update, dubbed Mayday by webmaster last month, happened to come up. According to Ryan Jones’ live blogging account of the SMX Keynote the update had nothing to do with the web spam team. It was an algorithmic change that was intended to “make long tail results more useful”. Matt made statements in effect telling webmasters who might have been affected by MayDay that they should look at their content and see how usefulness or unique content could be added to those pages. This indicates that the point of the Mayday update was to filter out or penalize results that are not unique content, or that are simply autogenerated results.

Matt made similar statements when he was interviewed by WebProNews and the topic came up:

How do I make sure that I am returning the highest quality content, stuff that’s really useful for users, whether it’s editorial discretion, unique content user generated content, you know, stuff that’s not available anywhere else, versus just something that’s scraped, or duplicate, or really kind of lower quality. – Matt Cutts, explaining how not to get penalized by the Mayday Update

During the keynote, in response to Matt’s explanation of what Mayday was supposed to accomplish Danny Sullivan indicated that he hopes that this update will help filter out results from content mills like Mahalo. Mahalo certainly sounds exactly like what Matt was describing, as I have written about in the past.

So, did the Mayday update actually accomplish filtering out this “low quality” content from the search results? I went back and checked some of Jason Calacanis’ spam pages on Mahalo that I had blogged about in the past. Unsurprisingly enough, most of the pages I checked were still ranking just fine in Google. What was slightly unexpected, however, was what the listing looked like for one of them, [need for speed walkthrough]:

Notice the lack of a snippet for that listing in the search results? That is because due to my earlier write-ups about Mahalo and Google, in an attempt to keep up appearances with Matt Cutts, Jason had the team move a bunch of the pages that I wrote about into a directory named /stub, and then blocked that directory via robots.txt. When Google encounters blocked content that has enough link juice it lists those pages, like you see here, as url-only. What Google usually doesn’t do, however, is actually rank those pages well in the search results.

Thinking that maybe it was a fluke, and that perhaps that particular listing just had some extra ranking power because I had blogged about it before and therefore it gained a few extra links, I delved further. This is just a sampling of what I found:

[fallen angel walkthrough] (#4)

[jack keane walkthrough] (#7)

[how to plan thanksgiving dinner] (#7)

[allheart coupons] (#11)

[adult friend finder coupons] (#1)

Let’s look at that, especially the last one.

This is a page with 0 (as in none, nil, nadda, zilch) spiderable content, yet Google has deemed it worthy of ranking it #1, above every other page in the index that matches that phrase. Every.Single.One. We’re given the company line telling webmasters that in order to succeed in ranking in Google one must focus on “quality” and “unique” content, yet Google decides to give Mahalo a golden ticket for pages they can’t even see?

Wtf?

Hm. Maybe the key is that if you want your duplicate, low quality, spammy content ranked then all you have to do is block it with robots.txt…?

Obviously that is not the case, and anyone who understands at all about how these things work will recognize that statement as ludicrous… but just to be sure, let’s look at some of the non-blocked content Mahalo pages are currently ranking for:

[julmust] (#7)

[The Alice B. Toklas Cookbook] (#11)

These Mahalo pages being returned are both examples of the many pages on Mahalo.com that are nothing more than content scraped directly from Wikipedia:

http://en.wikipedia.org/wiki/Julmust
http://en.wikipedia.org/wiki/The_Alice_B._Toklas_Cookbook

Note also that Wikipedia identifies some of their content as being “stubs” (the Alice B Toklas Cookbook page has a mere 261 words of content, including numbers and “a”, “and”, and “the”), but Mahalo is fine with presenting that exact same content as non-stub for whatever reason.

So, Danny, sorry… it looks as if Google did not achieve what it reportedly wanted to do with this latest update. It looks like both the low quality and completely duplicate (or even non-existant) content on Mahalo.com continues to rank. Maybe next time.

Jason Calacanis: Screw You Google, Now I’ll Sell Links Too

Michael VanDeMar — Fri, 12 Mar 2010 13:45:43 +0000

By now Google has to be getting more than a little embarrassed about the behavior of Mr. Jason Calacanis and his site, Mahalo.com. Aaron Wall did a very well written piece explaining how Mahalo Makes Black Look White and the spammy techniques they were employing. This isn’t the first time Aaron has blogged about Mahalo either, and talked about exactly how this makes Google look bad. For those who might not know, I have also been blogging about this recently.

While Google will ban smaller websites from their search results or from AdSense on a whim, usually it takes heavier coverage for bigger players to get hit. Like, for instance, when Google Blogoscoped outed BMW for spammy doorway pages. The story spread relatively fast, and Google wound up banning BMW for a short period of time. So when someone has a “special” relationship with Google, as Jason appears to have, and keeps getting second (and third, and fourth, and fifth…) chances to clean up their act, yet continues to snub their nose in Google’s general direction, it makes one wonder. Google has to be at least somewhat concerned that someone in the mainstream media will eventually notice and start to ask why someone like Jason would continually be allowed to get away with this stuff. Considering the unfairness and lack of impartiality of letting a site like Mahalo slide while punishing so many smaller sites for lesser offenses, my guess is that Google doesn’t actually want to discuss their reasons behind ignoring it. And so, each time Google does nothing, Jason decides to push things a little more.

This time it looks like Jason has decided to go ahead and violate the rules closest to Matt Cutts heart. While the layout I am showing here will change with time, since the header contains rotating articles, currently if you go to Mahalo at the top of every page (on the non-Answers side, anyways) you will see the following block of stories that Mahalo is highlighting (usually this area contains trending or hot news items):

(Click to enlarge)

While technically speaking just by looking at them there is nothing to distinguish one of those “featured stories” from another, the one that doesn’t actually belong is the third one in, with the caption “Best Pickup Line Ever?”. The reason that one is different from all the rest is simple… it’s not a Mahalo featured story at all, and has nothing to do with anything going on in the news. It’s an ad. It is a paid link that Mahalo.com sold, one that leads to a site built to market a wine company. There is nothing visual to distinguish or disclose that as an ad, and if we view the source of the page…

(Click to enlarge)

… we can see that there is nothing machine readable (ie. nofollow attribute) to distinguish it as an ad, either.

Matt Cutts has been very, very clear on his take on sites that sell links that pass PageRank, or ones that don’t disclose that they are in fact ads: they are spamming. No if, ands, or buts about it, they deserve to get punished. In fact, he has even gone to far to state that in his view undisclosed paid links violate FTC guidelines.

So, Matt, recently you put out a call for link spam reports, including “paid links that pass PageRank”. Really, though, is there any point in reporting Mahalo to you? Are you going to actually take action, or, like you have done with Jason’s spam in the past, will you continue to simply look the other way? Any other site would be faced with a penalize/ban first, make nice nice with Google later. Hell, with the BMW site you penalized them after they cleaned it up, just to make an example. I get the strangest feeling, though, that won’t happen with Mahalo…

Can you at least say something about this issue…?

Update: in response to a comment below and a question posed by Matt Cutts about what makes me believe that this is indeed a paid link:

The Mahalo Paid Link Evidence Trail.

Jason Calacanis’ Backup Plan For Replacing Content: Steal It From Wikipedia

Michael VanDeMar — Thu, 11 Mar 2010 18:11:08 +0000

/sigh

Ok Jason, we get it, you’re desperate. But stealing content from Wikipedia in order to replace what you deleted? Come on!

I am flipping through Mahalo.com today, just seeing if you’re keeping your word or not, when all of a sudden I notice this huge amount of pages with odd names that somehow I missed before:

http://www.mahalo.com/cgs-20625
http://www.mahalo.com/cgs-9896
http://www.mahalo.com/cp-154-526
http://www.mahalo.com/daa-1097
http://www.mahalo.com/daa-1106

These are all nothing more than content stolen from Wikipedia. Your version:

Wikipedia’s version:

You even hyperlinked the same internal linking scheme to the same topics Wikipedia does, regardless of whether or not those pages exist on Mahalo.com. How the hell can you claim this is original content when it is nothing more than cut and paste? I mean, wtf, you just claimed that Wikipedia is nothing more than a free for all a little under 3 weeks ago… does the content somehow take on some magical value after you scrape it and host in on your servers, trying to pass it off as something one of your users wrote? THIS is the “our users build it” content that you were referring to?

News flash, Jason… your users and Wikipedia’s users are not the same people. You lumped Squidoo into the same category back then as well. If I look close enough, will I find content stolen from them too?

Now, to be fair, maybe this content existed all along but was much, much less noticeable when you had all of those other pages of fluff in there, but now that you have deleted 78% of that side of Mahalo, these scraped pages are practically impossible to miss. None of the pages I looked at were actually indexed in Google, but it looks like at least 27,900 of them currently are:

(Click to enlarge)

Considering that Google only has a portion of those indexed, and that at last count there were only 128,324 pages left on that side of your site, that means that at minimum over 21% (and in all likelihood 30% – 40%) of the remaining pages on Mahalo.com are these scraped ones. Is that really what you want on your “Human Powered Search Engine”…?

There are 2 major differences between the original content and your version. 1) The original content cites the sources directly there on the page, whereas Mahalo does not, and 2) Mahalo is using each and every one of these scraped pages to automatically create 2 (and sometimes 3) additional contentless pages under the guise of questions being asked anonymously, questions that no one ever actually asked, but that are there solely for the purpose of bolstering your indexed page count in Google. The first question asked of every drug is What are the side effects of {insert drug}?:

And the second is always Where can I get {insert drug}:

I also saw a “Who makes {insert drug}” question here and there as well. These are empty questions, asked by a bot, that for the most part will never get answered (or even looked at) and were never intended to. Three plus free pages (two of them completely devoid of content) for the price of one scraped page. Jason, seriously, do you really think you are slick doing this?

By the way, a huge number of those pages are less than 100 words in length, yet none of them have the noindex tag. Regardless of of what the length is, however, it’s not what you are claiming Mahalo.com is Jason, it’s more fluff. Wikipedia already gives us those articles. You add nothing to the interwebs by copying them. In all seriousness you should just go through and delete them all.

Is Google Referrer Spamming Too Now?

Michael VanDeMar — Tue, 16 Feb 2010 13:59:41 +0000

Yesterday a friend of mine sent me a section of her traffic logs that were showing some odd information. According to what was recorded there her brand new, as of yet unlinked-to website was ranking on the first page of Google for the single keyword, [free]. If she actually had managed to rank for that phrase it would be an amazing feat to say the least. The competition for that single word is enormous. Unsurprisingly, when performing that actual search her site is nowhere to be found. The site in question is barely one week old, and hasn’t even been launched yet.

What is surprising, to me anyways, is that it appears that the traffic is actually coming from a bot at Google… a bot that is cloaked, sending fake referrers, and behaving in exactly the same manner as MSN’s referrer spamming bot that first showed up a little over 2 years ago. I blogged about it back then, as did many others. Eventually, after much feedback from the community, they did halt the referrer spam practice. It was a bad idea for them to do it in the first place, and quite a few webmasters were perturbed about it. Two years was too long for it to go on, but at least they did finally stop doing it.

Now it looks like Google, for some unfathomable reason, has decided to start doing the exact same thing. The entries in my friend’s traffic logs looked like this:

74.125.126.81 – - [14/Feb/2010:16:34:03 -0600] “GET / HTTP/1.1″ 200 19361 “http://www.google.com/search?hl=en&q=free&btnG=Google+Search&aq=f&oq=” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)”

72.14.192.3 – - [14/Feb/2010:16:36:28 -0600] “GET / HTTP/1.1″ 200 19361 “http://www.google.com/search?hl=en&q=free&btnG=Google+Search&aq=f&oq=” “Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)”

The IP’s in question definitely belong to Google (as can be seen here 74.125.126.81, and here 72.14.192.3). However, unlike normal Googlebot IP’s, these are not associated with the Google domain name via dns. For instance, if you do a host lookup on 66.249.71.233 you will see that it resolves to the hostname crawl-66-249-71-233.googlebot.com. The IP’s that the referrer spam is coming from do not resolve to any hostname. Presumably, going on the logic that MSN gave when they were first called out for doing this, the reason for not having a reverse dns associated with the IP’s is to hide the fact that they actually are from Google. Similarly the user-agent of these bots is being cloaked as well. Instead actually identifying as Googlebot, “Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)”, these bots are pretending to be an actual user using IE6 on Windows.

Unlike actual web surfers, Google, you have no expectation of privacy. When you are a bot, skulking around trying to disguise yourself as someone else is poor netiquette to say the least. I am not sure exactly what prompted you to start doing this, but you really should just stop.

Update:

Barry Schwartz of Search Engine Land contacted Google about this, and they replied back that this is indeed them performing cloaked spidering. However, according to them it is not being done for spam detection purposes, and the particular referrers used were in error:

Turns out, we were running an experiment to detect malware targeting Hot Trends queries related to the Haiti crisis. Because this experiment was developed in response to an urgent situation we moved quickly and as a result used an incorrect Google search referrer which we’re now working to fix. Thanks for calling this issue to our attention and we apologize for any confusion we may have caused.

Google Fails 5th Grade Math Test

Michael VanDeMar — Tue, 02 Feb 2010 01:58:32 +0000

So, I think I finally discovered the cause of global warming. No, for reals. From what I can tell, miss Mother Nature started using Google Calculator in helping her figure out what kind of weather she should serve up to us. Now, if she were trying to bake a cake, or perhaps get driving directions, I am sure Google would have worked just fine. But for doing math involving temperatures…? Not so much.

I was playing around with the functions on Google Calculator last week, when I noticed some of the calculations weren’t quite right. Maybe Michael Bolton from Office Space was involved in writing the Google Calculator app, and wound up putting a decimal in the wrong place, but something sure isn’t adding up.

For those that don’t know, water boils at 100 degrees Celsius and freezes at 0 degrees Celsius. In Fahrenheit the range is 212 degrees (boiling) to 32 degrees (freezing). Even for those of you who might not have remembered those figures off the top of your head, most of us did learn them fairly early in our academic careers… probably right around fourth or fifth grade. While those numbers may vary slightly under extreme pressures, for the most part they are pretty much standard. A simple search on Google will in fact verify that they are correct.

Overall I think that Google Calculator is a pretty cool tool. You can even type math in using English, and it will do it’s best to figure out how to interpret the numbers. Usually it does an excellent job. When I was playing with it the other day, however, I got this odd response:

Erm… no.

Way off. Not even in the same ballpark. Google did manage to group the numbers in a meaningful way, correctly guessing what I actually meant by that question, and yet somehow still came up with the wrong answer. If you can’t do it in your head, that should be (64F/2) = 32F = 0C, or water’s freezing point.

Maybe it just has a problem with temperatures on the low end, I thought, and that if I go the other way we might get better results:

Wow. That’s hot. Also wrong.

Apparently not. That one should be 106F * 2 = 212F = 100C, or the temperature at which water boils. Instead we get temperatures hotter than most home ovens can go.

To give Google the benefit of the doubt, I decided to try and take out the conversions altogether, and just let it do simple temperature calculations, staying in just one measurements system:

Not even close. Google fails.

Nope, still no math love from the search giant. I guess Google just needs to go back to school.