November 21, 2007
Google Deindexing Sites Because of Bad Robots.txt File?
There have been a lot of forum posts recently and even articles written regarding Google deindexing sites due to bad robots.txt files.
This one just smells funny.
This issue behind it was simple: Use the “noindex” directive, which is used in the robots meta tag, in your robots.txt by mistake and run the risk of having your site deindexed by Google.
I have tested this heavily, and have not been able to get Google to deindex any of my domains. I even went into my Google Webmaster Central account and ran through different directives to get an error and was unsuccessful.
My conclusions are this: Google will just ignore incorrect directives.
Filed under Google by Jerry West
Leave a Comment