Caching very large result sets

[eluser]Daniel H[/eluser]
What is the rule of thumb when it comes to caching very large results sets? With caching on, and selecting an entire table of c. 40000 records I get....

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 445 bytes) in .../system/database/drivers/mysql/mysql_result.php on line 162

I can obviously increase the memory limit but I'm not sure what should be the maximum I should ever set it to.

Obviously I would never need to query the full 40000 records, but I'm just curious as to the max limit you'd need to use should be...

Thanks.

[eluser]manilodisan[/eluser]
Caching has it's own limitations as well and both of them may result in a memory exhaust when using large amounts of data. I tested this also in the past and found that, when dealing with 1000+ records mysql is faster than CI's ability to read the cached results.

[eluser]Daniel H[/eluser]
Great - thanks for the info.