Help regarding robots.txt |
Somebody please suggest me a robots.txt file for my codeigniter project. Am not sure which are the folders that should not be crawled by robots on a codeigniter project.
Thank You.
Code: # www.robotstxt.org/ |
Welcome Guest, Not a member yet? Register Sign In |