Yandex robots correctly process robots.txt, if: The file size doesn't exceed 500 KB. It is a TXT file named "robots", robots.txt. The file is located in the root directory of the site.

robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.

Robots.txt is a text file webmasters create to instruct web robots (typically search engine robots) how to crawl pages on their website. The robots.txt file is part of the the robots exclusion protocol (REP)...