Personal Website of Yakov Shafranovich

« Playing with Google AdSense         To Evil: A Monthly Column »

FBI.GOV Content No Longer Hidden

About two months ago I wrote about the exclusion of FBI.GOV from search engines and The Internet Archive via the use of ROBOTS.TXT. Now it seems that it was removed and everything is back to normal.

Also, on an interesting sidenote, it seems that the Internet Archive does not delete content when a robots.txt file is posted but rather hides it. A quick check reveals lots of FBI.GOV content going back several years.

Tags: google —

Permalink | Trackback URL | This post has