removing index.php, funky stuff going on

[eluser]ferno[/eluser]
Ok, so I edited the config file and changed "index.php" to "" and edited my .htacess file to read:

RewriteEngine on
RewriteCond $1 !^(index\.php|images|robots\.txt)
RewriteRule ^(.*)$ /index.php/$1 [L]

but when I run my site, everytime I click on a link (to access a controller) the URL changes appropriately, but the home page is shown everytime.

So my URL is changing correctly, but the actual page is not changing at all.

Any suggestions?