Press enter to begin your search

robots.txt

The robots.txt file gives instructions about your site to web robots that crawl your site regularily for indexing; this is called The Robots Exclusion Protocol.