Thanks for sharing, if any one still confused with robots.txt rules then refer below URL and upload the .txt file to your root folder.
 
robotstxt.org/robotstxt.html