Robots.txt

A file created by webmasters that tells search engine bots not to visit certain pages of a site.