Yesterday, Smackdown noted that Wired put a robots.txt in place in response to spammers trying to use their wiki for spamalicious links. Oddly enough, that robots.txt also blocked everyone and everything, including Google and all the rest of the search engines, from the site. (Caveat: When I say “blocked everyone”, I’m referring to everyone who bothers to adhere to robots.txt).
