Managing Git and .gitignore for a CI library..??

Right now I have my PayPal CodeIgniter library in a Git repo with nothing but the library files themselves.  CI itself is not included in the project folder at all.  This makes it difficult for me to make quick changes and test them, though.

I'd like to create a new project locally that has CI fully installed, and then load my library into that, but I don't want Git to track CI itself.  I just want to track my library files only, so the GitHub repo would still only have the library files, but I could easily work locally.  

I was thinking I could just setup a .gitignore to ignore everything, and then un-ignore the library files/folders.  This is proving very difficult for some reason.  I've wasted lots of time on various forums and reading Git documentation on how to get this done with very sporadic results.

Is there any sort of a template or standard practice I can follow to easily manage my own CodeIgniter libraries in Git without simply including the entire CI structure with the library?

Any information on this would be greatly appreciated.  Thanks!