[eluser]ntheorist[/eluser]
@vang
i'm currently working with datamapper and i've been working on similar issues with memory consumption on large queries.
How are you loading your data into your views? When i'm working with a single Object, i typically load it directly, like
Code:
$post = new Post();
$post->where('id',2)->get();
$viewdata = $this->load->view('post',array('post'=>$post), TRUE);
Although yes when dealing with a large number of rows, the 'all' array can get just as large, and a lot of that has to do with the fact that each object in the array is an instance of Datamapper. If you load the all array into a view, you will be essentially copying the array of datamaps. One thing i've tried to use to sort out just the data is the _to_array() function.
Code:
$post = new Post();
$post->get(); // get all posts
foreach($post->all as $p)
{
$this->load->view('post',$p->_to_array());
}
this way, you're not copying the entire instance to the view, just the data. Plus the returned array will nicely insert the variable names, so each $p->title becomes $title in your view.
Next, if you're using a template library or something that stores and 'renders' the view data, that process may also be making copies of your data, which multiplies with each row you send to it.
I struggled for a bit because i wanted to use datatables/tablesorter, but i opted to have sorting/paging done on the server side (with ajax as an option). The client side functionality is nice, but yes if your results grow to 1000+ rows etc, you'll be faced with either huge memory consumption by the sheer amount of data (at least on first run if you have caching on) and then have to resort to server side paging anyway at some point.
Lastly, i would recommend using some sort of opcode caching utility if you are not already, such as eaccellerator. I was delighted to see my scripts reduce memory consumption from 3+ megs to 500k at the flip of that switch.
cheers,
CC
