I think CI's query builder is pretty fast, and certainly not much different than using plain PHP. There are a lot of benefits when using the query builder. Read the documentation and dig around in the code to see for yourself.
One thing is for sure is that PHP itself has a pretty slow connection to MySQL. For instance, I had a batch of inserts that each included many megabytes of data, and PHP couldn't handle it. Running the batch through bash (linux terminal) was completed almost instantly. Seriously, I thought there must be an error because the bash script ran so fast, but there wasn't one.
The truth is that CI's query builder is super easy to use, and you'll see that it's plenty fast for most queries. It's more likely that you would design your application poorly, contributing to the overall inefficiency, rather than CI causing issues for you.
Take for example WordPress, which is extremely slow. If you look at the MySQL query log, even the most basic of pages causes almost 30 queries to run. Compare that to a well designed application where queries are optimized, and database is normalized, and you will not likely have any reason to complain.
