In all of the discussions and posts about Jason Calacanis, whenever and wherever he replied to people calling him on his bullshit, it always had the same tired familiar ring to it. His statements have that tone that all scam artists and con men have utilized throughout the ages, professing their innocence despite the preponderance of evidence against them. Finally it hit me who it is that he sounds like when he is trying to defend his spammy sites…
zOMG! Jason Calacanis Lied Again?? Shocker!
Last Thursday, in response to Matt Cutts stating that he needed more than “arbitrary inurl searches” to sway him (which was in turn in response to a Hacker News submission about Mahalo and the plethora of keyword rich domains they were apparently building out) I wrote a post explaining in some detail how the latest Mahalo spam is in fact spam. I demonstrated in the post how Jason had developed a linkfarm which was being used as a link source back to Mahalo.com. It wasn’t just that the individual sites were all linking back to the mother site, which would in fact be normal, but also that the pages were linking back to specific pages within the main site, pages that in many cases had few, if any, links going to them aside from the ones from this linkfarm.
Each time it happens Matt’s defense of Mahalo spamming Google just gets more perplexing. In this latest round he started by saying that his job was not to have knee jerk reactions, as if Mahalo hadn’t already established a pattern of spamming over a long period of time, and that Matt is pretending he hadn’t already had a talk with Jason and told him that if he didn’t raise the bar with his site that Google would take action on Mahalo. From there it got even weirder – Matt looked at the linkfarm and basically told me that a) he didn’t care as long as it wasn’t passing link juice, and b) he’s the only one who could tell if that was the case.
I could have sworn that it was if you were caught trying to spam you were penalized, and you couldn’t get the penalty removed unless you promised not to do it again. Now, where did I get such a crazy and wild idea? Oh yeah, I remember now…
Need Help Understanding The Latest Mahalo Spam?
On Tuesday of this week someone posted the following question to the Hacker News website: How long has Mahalo been using keyword domains like this? The link in the story points to a search in Google, [inurl:tip_guidelines mahalo]. The results of this query show a list of somewhere between 180 and 270 sites (Google doesn’t show all of them, just the first 184 or so) all belonging to Mahalo.com, all keyword rich domains, all using the Mahalo Answers platform, and all covering material that Mahalo.com already covers. I am sure most of you are familiar with that fact that Google labels sites that have little or no content and are designed to drive affiliate conversions as Thin Affiliate sites:
These sites usually have no original content and may be cookie-cutter sites or templates with no unique content. – Google Webmasters Tools Help, on sites Google does not like
These sites that Mahalo has started churning out, all that were apparently created just this year, would appear to be the AdSense version of the classic “thin affiliate” website.
I showed Matt Cutts the link to the search itself, and asked if he thought that the list of sites
Dear Matt Cutts, What’s Your Take On Addon Domains?
Today Matt Cutts answered a question from “Land Lubber”, Colorado. Land Lubber asks:
What’s your take on “addon domains”? Does Google penalize someone for having one or more addon domains on their main website, (or if they’re self hosting)? e.g. 2, 5, or 10 all coming from the same IP address, would that be bad? – Land Lubber, CO
Matt responded with the following:
Rackspace Hacked Clients, Check Your Databases: WordPress “wp_optimize” Backdoor In wp_options Table
Just finished cleaning up a hacked client whose website is hosted on Rackspace Cloud hosting. It is the second one within the past few weeks, although the first one was actually hosting on Laughing Squid, which happens to use Rackspace Cloud. I had discovered that there were a large number of people all on the same IP as my client a couple of weeks ago who all got hacked, but I was having trouble determining if it was an issue with Laughing Squid or an issue with Rackspace Cloud itself, so I didn’t blog about it until I could research it more. I wish now that I had, because maybe then it would not have spread so widely. As it is, it is the same WordPress attack that Unmask Parasites blogged about earlier today.
It looks like the culprit might have been a security hole in phpmyadmin. Hopefully this will turn out to be what was wrong,
Was The Google Mayday Update A Complete Failure Then?
Earlier this week at SMX Advanced Seattle, during the You&A With Matt Cutts, the topic of the latest Google update, dubbed Mayday by webmaster last month, happened to come up. According to Ryan Jones’ live blogging account of the SMX Keynote the update had nothing to do with the web spam team. It was an algorithmic change that was intended to “make long tail results more useful”. Matt made statements in effect telling webmasters who might have been affected by MayDay that they should look at their content and see how usefulness or unique content could be added to those pages. This indicates that the point of the Mayday update was to filter out or penalize results that are not unique content, or that are simply autogenerated results.
Matt made similar statements when he was interviewed by WebProNews and the topic came up:
WordPress Hacking, Matt Mullenweg, And Some Screwed Up Priorities
I clean WordPress installations for people who have been hacked. I can help fix non-Wordpress sites as well, but since often times the way people find me is through the guide I wrote on how to fix WordPress after you’ve been hacked it turns out that’s what they need me to do for them a fair bit of the time. I have a process that I go through, and a specific set of things that I look for on every WordPress installation that I work on to make sure that it is indeed hacked, and to determine how bad the damage is. Different intrusions can leave various symptoms and clues as to how the hacker got in, and knowing this can be helpful in diagnosing the situation.
One of the hacks that has been around for a few years
My Mom Needed Me To Let The Plumber In While She Was At Work (True Story)
I work from my house and keep odd hours, so when a family member needs some sort of worker let into their house during the day I am often asked if I am available to do it. I don’t mind, we all live fairly close together, and it’s not that much of a hassle on most days. Tonight my mom called and asked me if I could let someone in to her place tomorrow to look at her tub, because it’s clogged. She’s tried Drano twice, poured boiling hot water in it, and even tried plunging it, all to no avail. I told her it would be no problem for me to let someone in.
A little while later I went into my own bathroom, and while in there happened to glance at my own tub…
GoDaddy’s Suggestion For The Cause Of Their Hacks And Their Community Blog – Can You Smell The Irony?
Yesterday I blogged about the hacking situation with GoDaddy hosting and a customer service call I had with them concerning some evidence I had found. While it is true that as this has progressed GoDaddy has widened their scope in investigating what the underlying cause of these hacks are, initially they claimed that the issue was with their customers running outdated versions of WordPress. While being wrong about something like that is usually not that big of a deal, in this particular instance it proved to be beyond irksome, since a large portion of their customer base were told that it was their own fault that their sites got hacked (even in cases where the customer was up to date), and that GoDaddy was in no way to blame:
WordPress is a-ok. Go Daddy is rock solid. Neither were ‘hacked,’ as some have speculated.
After an extensive investigation, we can report there was a small group of customers negatively impacted. What happened? Those users had outdated versions of the popular blogging software, set up in a particular way. – Alicia from GoDaddy
From what I have read around the web customers were being told that it was not GoDaddy’s responsibility to fix the sites, that they only offered “limited support” in situations like this, leaving people with only the option of restoring from a backup (which would often not help even in outdated WordPress hack situations, since hacks can go undetected for months) or hiring outside help to clean things up.
You can see on the support page they have set up, What’s Up with Go Daddy, WordPress, PHP Exploits and Malware? that they still claim that outdated scripts are part of the problem. Going to that page and viewing the source reveals something almost unbelievable:
(click to enlarge)
That’s right, in a classic “do as I say, not as I do” twist it seems that GoDaddy is in fact running an older version of WordPress (WordPress MU, based on the version number, which has the same security holes as regular WordPress) for their community blog that they are using to tell people to upgrade their WordPress versions.
To be fair, simply having an older version of WordPress does not mean that it is automatically insecure… the security fixes in the more recent versions may be minor and the known vulnerabilities might have been manually patched. I can’t know without actually digging deeper and looking if in fact the installation was vulnerable.
Then again… neither can GoDaddy in the case of their customers.
Hosting With GoDaddy? Might Want To Rethink That Decision.
One of the services I offer people is cleaning their WordPress installations of hacks and infections, mostly for those who might not have the time or technical expertise to follow my hacked WordPress cleaning guide. Therefore when something happens that increases the number of people getting hacked, such as when a new exploit is discovered, or a security hole in a large host starts getting exploited (like what happened with Network Solutions last month), I get an increase in the number of people requesting help cleaning things up. This month it started happening with a large number of GoDaddy customers.
When it first started to happen I did some searching around, and noticed that there was some discussion going on about the heightened GoDaddy hacking activity, but at that time everything I read that stated the problem was with GoDaddy customers all had roots pointing back to a single post on a company blog that didn’t offer enough details for me to really see why it was happening there and not other places. Not that WordPress on other hosts weren’t still getting hacked, but there has definitely been a higher concentration of instances on GoDaddy. GoDaddy was definitely aware of the issue, and even replied in some threads on the WordPress.org help forum:
GoDaddy.com did send out a notification to customers affected by this issue. Although I know you would prefer not to be linked, I want to avoid flooding the forum. For a step-by-step guide to update WordPress, please visit http://fwd4.me/NGN – Alicia from GoDaddy.com
The link to their “step-by-step guide” to updating WordPress turns out to be nothing more than than a link back to WordPress’ own guide to upgrading, and links on how to back up your stuff on GoDaddy. Decidedly not step-by-step imo, and in this case not all that helpful. If the reason your site gets hacked is due to you running an older, insecure version of WordPress, once that happens simply upgrading will not fix the issue. This seems to me to be a bit of a lame response to a serious issue coming from a company that bills itself as the “World’s largest Hosting Provider”.
GoDaddy keeps insisting that the problem is due to outdated WordPress installations, and that staying up to date and site security is the responsibility of the customer, not of GoDaddy. In one sense I completely agree with them. If you run an older version of WordPress that has known security holes in it (ie. pretty much all versions aside from the most recent) then the odds are that you are going to get hacked. Most of the clients I cleaned from GoDaddy so far were up to date, running version 2.9.2, but this still didn’t mean that it was GoDaddy’s fault, since it is possible for a site to get hacked and no signs show up for months. This means that the sites I was cleaning could potentially have had the hack from an older version, and it only became apparent some time after they upgraded.
The problem is that after doing some very thorough clean up jobs (ie. wipe and reinstall), and making sure the clients were up to date, all passwords changed, all image files verified as actual images, clean WordPress, clean theme, clean plugins, and hand cleaning the database, I had clients still getting re-hacked.
One client I had was having issues with funky characters in his posts. He would make the post, everything would be fine, and then the next day they would be converted in a way that would make them display as unicode. This was well after I had done my cleaning, and no one should have made any changes to the database since then. My assumption was that GoDaddy themselves was making changes, possibly security upgrades related to the recent hacking waves, and I figured that calling them to see what they had done would be the best bet. In preparation for this I went ahead and logged into the client’s account, and ftp’d into the server just to make sure everything looked like it was in place still. As soon as I did I saw that about 30 minutes before a brand new, non-Wordpress, oddly named php file had been dropped into my client’s site.
I downloaded the file and looked at it. I suddenly realized that this was the source file for all of the hacks that were happening. It was named “plan_erich.php”, and had similar eval(base64_decode( instruction at the top of the file. I modified the code to be able to decrypt it safely, and looked through the output (which you can view here). The script was designed to delete itself as soon as it ran:
$z=$_SERVER["SCRIPT_FILENAME"];
@unlink($z);
Finding this script before it was triggered and deleted itself was raw luck. Catching this file gave a great opportunity to actually track down how these hacks are occurring, and possibly would leave clues that GoDaddy could use to keep it from happening again. Looking at the owner/creator of the file, and matching that timestamp up with the various logs (ftp, ssh, http, mysql, etc) could give GoDaddy the information needed to figure out how the file really got there, instead of just guessing that WordPress was the issue. I have never seen a file like this before, and searching Google for the name yielded no results, so there really was no other information out there available on this. Finding it there was a little like hitting the lottery in that respect, random and very, very good luck.
The problem, however, is that GoDaddy didn’t seem to care. I called and explained to the woman I spoke with exactly what it was that I found and how it could be useful. I told her that matching up that file to the logs could yield some potentially valuable information. She did listen carefully, and I am pretty sure she understood what I was saying, because she asked if she could put me on hold to go talk with someone who might know more. She came back and informed me that she didn’t have permission to look at those logs.
I explained again, in a little more detail, why looking at the section of those logs was very important, and if she didn’t have permission could she please escalate the ticket to someone who did. Again, she put me on hold. This time she came back and told me that they were uninterested in escalating it.
At this point I was a teensy bit amazed at GoDaddy’s lack of concern with the issue. She very kindly informed me that the issue was that the client was running an older version of WordPress, and that we needed to upgrade. Wtf? I went and looked, and made sure that he was indeed still running the 2.9.2 version that I had installed over a week ago (and remember, he was running that version before I ever did anything), and he was. I told her that. She told me that no, she was looking at what the hosting control panel said, and that he was running version 2.6.
That was when it struck me… GoDaddy was claiming that this wave of WordPress hacks was due to clients not upgrading without even bothering to really look at the clients sites. The hosting control panel can only report what was installed via the hosting control panel itself. If a client pushes the button to upgrade WordPress from within the WordPress admin section then the hosting control panel will never know.
As amazing as it seems, apparently the entire GoDaddy technical support team is ignorant of this fact. That’s right… the “World’s largest Hosting Provider” doesn’t understand the very basics of how the world’s largest blogging platform works.
Something, probably a hosting configuration, is allowing GoDaddy customers to have their sites hacked, and it isn’t file permissions, insecure passwords, or out of date software. Not being willing to even look when a developer calls to tell you that they found something is completely unacceptable. My suggestion to all GoDaddy hosting customers: bail now, before something happens to your site. This is not a WordPress issue only… although it seems to have targeted WordPress customers first, all sites that use php are at risk. Personally for shared hosting I recommend Hostgator, because I love their tech support (and their servers are very robust), but there are plenty of hosts out there to choose from (Disclosure: I changed the previous link to an affiliate link, although if you’d rather purchase hosting from them without giving me credit that’s fine too, here is a clean link for you: HostGator).
Bob Parsons, I am sorry. Hot chicks and a strong tits and ass marketing campaign do not make up for apathy in matters of client security and well being.
