How to restrict IP on folder of controllers from apache config?

[eluser]gwerner[/eluser]
I'd like to restrict access to a folder of controllers that are used for admin purposes only. I've tried a number of ways and not coming up with a solution. These controllers are behind password protection. But, I'd like to just remove it from view if someone happens to stumble upon the right directory. Can this be done? I'd rather not do it from htaccess. I have access to the apache config files, so I'd like to handle it there.

Does it have anything to do with the way Codeigniter routes? Or, am I just way off?

This what I'm using that doesn't work

Code:

<Directory /var/www/application/controllers/folder/>
Order deny,allow
Deny from all
</Directory>