+- CodeIgniter Forums (https://forum.codeigniter.com)
+-- Forum: Archived Discussions (https://forum.codeigniter.com/forumdisplay.php?fid=20)
+--- Forum: Archived Development & Programming (https://forum.codeigniter.com/forumdisplay.php?fid=23)
+--- Thread: robots.txt paths (/showthread.php?tid=13925)
robots.txt paths - El Forum - 12-11-2008
[eluser]Unknown[/eluser]
Hi,
When creating a robots.txt file for a site coded with CI, is it proper to include "/index.php/" in the paths inside the robot.txt file?
User-agent: *
Disallow: /index.php/admin
Thanks.
robots.txt paths - El Forum - 12-11-2008
[eluser]gstjohn[/eluser]
You should be fine including it. The only reason you wouldn't need it is if you are using mod_rewrite to remove 'index.php' from the URL.