CodeIgniter Forums
robots.txt paths - Printable Version

+- CodeIgniter Forums (
+-- Forum: Archived Discussions (
+--- Forum: Archived Development & Programming (
+--- Thread: robots.txt paths (/showthread.php?tid=13925)

robots.txt paths - El Forum - 12-11-2008

When creating a robots.txt file for a site coded with CI, is it proper to include "/index.php/" in the paths inside the robot.txt file?

User-agent: *
Disallow: /index.php/admin


robots.txt paths - El Forum - 12-11-2008

You should be fine including it. The only reason you wouldn't need it is if you are using mod_rewrite to remove 'index.php' from the URL.