When should I autoload a library?

[eluser]nyxynyx[/eluser]
Hi, I'm wondering when I should autoload libraries and when should I load it in the controller when required? If I use a particular library on 50% of the pages viewed daily, should I autoload it? Is there a significant performance impact of autoloading libraries? Sorry I'm new to this