My Mom Needed Me To Let The Plumber In While She Was At Work (True Story)

Complex bath mechanisms I work from my house and keep odd hours, so when a family member needs some sort of worker let into their house during the day I am often asked if I am available to do it. I don’t mind, we all live fairly close together, and it’s not that much of a hassle on most days. Tonight my mom called and asked me if I could let someone in to her place tomorrow to look at her tub, because it’s clogged. She’s tried Drano twice, poured boiling hot water in it, and even tried plunging it, all to no avail. I told her it would be no problem for me to let someone in.

A little while later I went into my own bathroom, and while in there happened to glance at my own tub…

Read more

GoDaddy’s Suggestion For The Cause Of Their Hacks And Their Community Blog – Can You Smell The Irony?

Yesterday I blogged about the hacking situation with GoDaddy hosting and a customer service call I had with them concerning some evidence I had found. While it is true that as this has progressed GoDaddy has widened their scope in investigating what the underlying cause of these hacks are, initially they claimed that the issue was with their customers running outdated versions of WordPress. While being wrong about something like that is usually not that big of a deal, in this particular instance it proved to be beyond irksome, since a large portion of their customer base were told that it was their own fault that their sites got hacked (even in cases where the customer was up to date), and that GoDaddy was in no way to blame:

WordPress is a-ok. Go Daddy is rock solid. Neither were ‘hacked,’ as some have speculated.

After an extensive investigation, we can report there was a small group of customers negatively impacted. What happened? Those users had outdated versions of the popular blogging software, set up in a particular way. – Alicia from GoDaddy

From what I have read around the web customers were being told that it was not GoDaddy’s responsibility to fix the sites, that they only offered “limited support” in situations like this, leaving people with only the option of restoring from a backup (which would often not help even in outdated WordPress hack situations, since hacks can go undetected for months) or hiring outside help to clean things up.

You can see on the support page they have set up, What’s Up with Go Daddy, WordPress, PHP Exploits and Malware? that they still claim that outdated scripts are part of the problem. Going to that page and viewing the source reveals something almost unbelievable:

GoDaddy outdated software...?
(click to enlarge)

That’s right, in a classic “do as I say, not as I do” twist it seems that GoDaddy is in fact running an older version of WordPress (WordPress MU, based on the version number, which has the same security holes as regular WordPress) for their community blog that they are using to tell people to upgrade their WordPress versions.

To be fair, simply having an older version of WordPress does not mean that it is automatically insecure… the security fixes in the more recent versions may be minor and the known vulnerabilities might have been manually patched. I can’t know without actually digging deeper and looking if in fact the installation was vulnerable.

Then again… neither can GoDaddy in the case of their customers.

Hosting With GoDaddy? Might Want To Rethink That Decision.

One of the services I offer people is cleaning their WordPress installations of hacks and infections, mostly for those who might not have the time or technical expertise to follow my hacked WordPress cleaning guide. Therefore when something happens that increases the number of people getting hacked, such as when a new exploit is discovered, or a security hole in a large host starts getting exploited (like what happened with Network Solutions last month), I get an increase in the number of people requesting help cleaning things up. This month it started happening with a large number of GoDaddy customers.

When it first started to happen I did some searching around, and noticed that there was some discussion going on about the heightened GoDaddy hacking activity, but at that time everything I read that stated the problem was with GoDaddy customers all had roots pointing back to a single post on a company blog that didn’t offer enough details for me to really see why it was happening there and not other places. Not that WordPress on other hosts weren’t still getting hacked, but there has definitely been a higher concentration of instances on GoDaddy. GoDaddy was definitely aware of the issue, and even replied in some threads on the WordPress.org help forum:

GoDaddy.com did send out a notification to customers affected by this issue. Although I know you would prefer not to be linked, I want to avoid flooding the forum. For a step-by-step guide to update WordPress, please visit http://fwd4.me/NGNAlicia from GoDaddy.com

The link to their “step-by-step guide” to updating WordPress turns out to be nothing more than than a link back to WordPress’ own guide to upgrading, and links on how to back up your stuff on GoDaddy. Decidedly not step-by-step imo, and in this case not all that helpful. If the reason your site gets hacked is due to you running an older, insecure version of WordPress, once that happens simply upgrading will not fix the issue. This seems to me to be a bit of a lame response to a serious issue coming from a company that bills itself as the “World’s largest Hosting Provider”.

GoDaddy keeps insisting that the problem is due to outdated WordPress installations, and that staying up to date and site security is the responsibility of the customer, not of GoDaddy. In one sense I completely agree with them. If you run an older version of WordPress that has known security holes in it (ie. pretty much all versions aside from the most recent) then the odds are that you are going to get hacked. Most of the clients I cleaned from GoDaddy so far were up to date, running version 2.9.2, but this still didn’t mean that it was GoDaddy’s fault, since it is possible for a site to get hacked and no signs show up for months. This means that the sites I was cleaning could potentially have had the hack from an older version, and it only became apparent some time after they upgraded.

The problem is that after doing some very thorough clean up jobs (ie. wipe and reinstall), and making sure the clients were up to date, all passwords changed, all image files verified as actual images, clean WordPress, clean theme, clean plugins, and hand cleaning the database, I had clients still getting re-hacked.

One client I had was having issues with funky characters in his posts. He would make the post, everything would be fine, and then the next day they would be converted in a way that would make them display as unicode. This was well after I had done my cleaning, and no one should have made any changes to the database since then. My assumption was that GoDaddy themselves was making changes, possibly security upgrades related to the recent hacking waves, and I figured that calling them to see what they had done would be the best bet. In preparation for this I went ahead and logged into the client’s account, and ftp’d into the server just to make sure everything looked like it was in place still. As soon as I did I saw that about 30 minutes before a brand new, non-Wordpress, oddly named php file had been dropped into my client’s site.

I downloaded the file and looked at it. I suddenly realized that this was the source file for all of the hacks that were happening. It was named “plan_erich.php”, and had similar eval(base64_decode( instruction at the top of the file. I modified the code to be able to decrypt it safely, and looked through the output (which you can view here). The script was designed to delete itself as soon as it ran:


$z=$_SERVER["SCRIPT_FILENAME"];
@unlink($z);

Finding this script before it was triggered and deleted itself was raw luck. Catching this file gave a great opportunity to actually track down how these hacks are occurring, and possibly would leave clues that GoDaddy could use to keep it from happening again. Looking at the owner/creator of the file, and matching that timestamp up with the various logs (ftp, ssh, http, mysql, etc) could give GoDaddy the information needed to figure out how the file really got there, instead of just guessing that WordPress was the issue. I have never seen a file like this before, and searching Google for the name yielded no results, so there really was no other information out there available on this. Finding it there was a little like hitting the lottery in that respect, random and very, very good luck.

The problem, however, is that GoDaddy didn’t seem to care. I called and explained to the woman I spoke with exactly what it was that I found and how it could be useful. I told her that matching up that file to the logs could yield some potentially valuable information. She did listen carefully, and I am pretty sure she understood what I was saying, because she asked if she could put me on hold to go talk with someone who might know more. She came back and informed me that she didn’t have permission to look at those logs.

I explained again, in a little more detail, why looking at the section of those logs was very important, and if she didn’t have permission could she please escalate the ticket to someone who did. Again, she put me on hold. This time she came back and told me that they were uninterested in escalating it.

At this point I was a teensy bit amazed at GoDaddy’s lack of concern with the issue. She very kindly informed me that the issue was that the client was running an older version of WordPress, and that we needed to upgrade. Wtf? I went and looked, and made sure that he was indeed still running the 2.9.2 version that I had installed over a week ago (and remember, he was running that version before I ever did anything), and he was. I told her that. She told me that no, she was looking at what the hosting control panel said, and that he was running version 2.6.

That was when it struck me… GoDaddy was claiming that this wave of WordPress hacks was due to clients not upgrading without even bothering to really look at the clients sites. The hosting control panel can only report what was installed via the hosting control panel itself. If a client pushes the button to upgrade WordPress from within the WordPress admin section then the hosting control panel will never know.

As amazing as it seems, apparently the entire GoDaddy technical support team is ignorant of this fact. That’s right… the “World’s largest Hosting Provider” doesn’t understand the very basics of how the world’s largest blogging platform works.

Something, probably a hosting configuration, is allowing GoDaddy customers to have their sites hacked, and it isn’t file permissions, insecure passwords, or out of date software. Not being willing to even look when a developer calls to tell you that they found something is completely unacceptable. My suggestion to all GoDaddy hosting customers: bail now, before something happens to your site. This is not a WordPress issue only… although it seems to have targeted WordPress customers first, all sites that use php are at risk. Personally for shared hosting I recommend Hostgator, because I love their tech support (and their servers are very robust), but there are plenty of hosts out there to choose from (Disclosure: I changed the previous link to an affiliate link, although if you’d rather purchase hosting from them without giving me credit that’s fine too, here is a clean link for you: HostGator).

Bob Parsons, I am sorry. Hot chicks and a strong tits and ass marketing campaign do not make up for apathy in matters of client security and well being.