MVC for long lists of items -- found no examples

[eluser]atlas7[/eluser]
All the examples of design using the MVC (model-view-controller) approach that I have seen so far involve the controller getting data from a model into one or more variables, and then feeding those variables to a view for display. So if we wish to display a very long web page containing 10,000 items, the controller will first get those 10,000 items into an array by calling a function within a model, and then it will feed that array to a view for display. This means that we must allocate enough memory to essentially hold everything that will end up on a web page before we send it to the http client.

The more memory-efficient approach seems to be not generally discussed. This would involve doing a database query, reading the result one row at a time, and then sending that row to the http client until the entire list has been sent. You can of course do this with unstructured PHP. But how to you do this gracefully within the MVC approach? I have looked around for examples of this in the PHP world and found none.

Note that using pagination isn't the answer because it answers a different question.

[eluser]GSV Sleeper Service[/eluser]
I don't understand. What you are describing sounds like pagination to me. How is this different to pagination?

[eluser]xwero[/eluser]
you could use a silent paging trick, you described, in your controller.

Code:

$limit = 100;
$offsets = range(0,$this->model->count_rows(),$limit);
for($offsets as $offset)
{
    $data['items'] = $this->model->get_items($limit,$offset);
    $this->load->view('rows', $data);
}

Cutting up the data that is queried reduces the memory needed to move it to php and by using free_result the memory is flushed.
By outputting the data to the browser, instead of storing the output in strings, the view method takes care of flushing the php memory.

I wouldn't go as far as querying one row at the time because sql to php functions are slow. You have to find a balance between speed and memory efficiency.

[eluser]atlas7[/eluser]
The silent paging technique seems promising, thanks. However, each call to the view will only send a partial page, right? So the controller will first have to call a view to send the initial part of the page (doctype, head, header, etc.), then it will call the view with each set of rows as in your example, and then finally it will have to call the view to tell it to display the rest of the page.

I can see this getting quite messy. The controller now has to have knowledge about how the page is being constructed, and this defeats the MVC separation.

Perhaps the controller should send a list of indices to the view, and the view should then call the model to get the data items corresponding to the indices and display them.

I would be very interested in seeing examples of how any of this has been cleanly implemented by anybody else.

[eluser]xwero[/eluser]
That the controller knows how the page is constructed is not against the MVC pattern because it's his job to take page constructing decisions.

If you are working with a template working library i agree this can become ugly but that is the disadvantage of the solution. But if you want to display that many records you can get creative and split the template in a pre and post rows part.

[eluser]Developer13[/eluser]
As long as your model has been loaded inside your controller, you can query against your model directly from your view if that helps.

[eluser]atlas7[/eluser]

That the controller knows how the page is constructed is not against the MVC pattern because it's his job to take page constructing decisions.

No, this is wrong. If the controller contains any awareness even of the fact that output is to a web page, then MVC has no meaning, and it becomes impossible to use a different view to send output to some other type of device.