Welcome Guest, Not a member yet? Register   Sign In
robots.txt paths
#1

[eluser]Unknown[/eluser]
Hi,
When creating a robots.txt file for a site coded with CI, is it proper to include "/index.php/" in the paths inside the robot.txt file?

User-agent: *
Disallow: /index.php/admin

Thanks.
#2

[eluser]gstjohn[/eluser]
You should be fine including it. The only reason you wouldn't need it is if you are using mod_rewrite to remove 'index.php' from the URL.




Theme © iAndrew 2016 - Forum software by © MyBB