index.php automatically being added to links?

Hi

This is my first experience with CI4 after being comfortable with 3. As with any fresh CI installation I've spent the last chunk of time removing index.php from the URLs (do people use this? why is it enabled by default?). 

I've got to the point where it is almost done - i.e. if i navigate to mysite.com/controller, everything functions as expected. However if I link to /controller/ on my page, when clicking it always redirets to index.php/controller. 

Now from my setting up this still brings back the correct page but it's a horrendous looking URL. Does anybody know where the settings/ how I can remove index.php from being automatically set on my URLs? The link itself still claims to go to /controller/ so I'm assuming there's some kind of routing there automatically redirects there or something? I'm not a fan so any help would be appreciated

thanks