XML-RPC memory leak question..

[eluser]ArcticZero[/eluser]
I'm currently using Adodb Lite 1.42 (latest), but from what I've seen, the problem seems to occur during the actual sending/receiving of parsed data.

[eluser]TheFuzzy0ne[/eluser]
My findings so far suggest that the problem is in this part of your code:

Code:

foreach($sales_items as $item)
{
    log_message('error', $x);
    $item_array = array();
    if(isset($item['id']))
    {
        $item_array['item_id'] = new xmlrpcval($item['item_id'], 'string');
        $item_array['quantity_purchased'] = new xmlrpcval($item['quantity_purchased'], 'string');
        $item_array['item_unit_price'] = new xmlrpcval($item['item_unit_price'], 'string');
        $item_array['item_buy_price'] = new xmlrpcval($item['item_buy_price'], 'string');
        $item_array['item_tax_percent'] = new xmlrpcval($item['item_tax_percent'], 'string');
        $item_array['item_total_tax'] = new xmlrpcval($item['item_total_tax'], 'string');
        $item_array['item_total_cost'] = new xmlrpcval($item['item_total_cost'], 'string');
        $items_array[$x] = new xmlrpcval($item_array, 'struct');
        $x++;
    }
}

As you can see, I've added a simple log_message call in there, and my log shows that $x only gets as far as being incremented to 9, that's 10 iterations in total, and it chokes on the 11th.

I'm still looking into it for you, but I thought I'd share my findings so far in case the solution becomes apparent to you, or anyone else.

[eluser]TheFuzzy0ne[/eluser]
The problem looks like it's recursion.

This line:

Code:

$items_array[$x] = new xmlrpcval($item_array, 'struct');

is the line I think is responsible.

You might want to try something like:

Code:

foreach($sales_items as $item)
{
    log_message('error', $x);
    $item_array = array();
    $tmp_arr = array();
    if(isset($item['id']))
    {
        $tmp_arr['item_id'] = new xmlrpcval($item['item_id'], 'string');
        $tmp_arr['quantity_purchased'] = new xmlrpcval($item['quantity_purchased'], 'string');
        $tmp_arr['item_unit_price'] = new xmlrpcval($item['item_unit_price'], 'string');
        $tmp_arr['item_buy_price'] = new xmlrpcval($item['item_buy_price'], 'string');
        $tmp_arr['item_tax_percent'] = new xmlrpcval($item['item_tax_percent'], 'string');
        $tmp_arr['item_total_tax'] = new xmlrpcval($item['item_total_tax'], 'string');
        $tmp_arr['item_total_cost'] = new xmlrpcval($item['item_total_cost'], 'string');
        $items_array[$x] = new xmlrpcval($tmp_arr, 'struct');
        $x++;
    }
}

[eluser]ArcticZero[/eluser]
Thanks! I've just tried slipping in an "echo '<br>' . $x;" in place of log_message, since the client itself doesn't run under CI. And it seems to be incrementing just fine, once for each item within a sale, then resetting to zero when it has to process items for the next row of sales.

EDIT: Alright, I'll try that modification and let you know my results.

[eluser]TheFuzzy0ne[/eluser]
I was wrong. In both versions of the code, the line of code responsible for the error appears to be this one:

Code:

$item_array['item_tax_percent'] = new xmlrpcval($item['item_tax_percent'], 'string');

...Still looking into it.

[eluser]ArcticZero[/eluser]
If you need it, here's a dump of the actual sample data I'm trying to send:

http://www.yousendit.com/download/U0d4OG...QndLSkE9PQ

I'll be looking in that recursion and see what I can do in the meantime.

[eluser]TheFuzzy0ne[/eluser]
I don't think this is a bug as such, I just think that it really is using a lot of memory. I noticed it seemed to choke when it got to iteration 727, so I stopped it there and print_r()d the $final_array. Even IE choked and hung... 5 minutes later, I can't even get the IE window to display, so I had to kill IE. There is a lot of data there.
I'm wondering if there's any way that you can write to a file, say every 100 iterations of the main loop, and when done, load the file and send that.

There may be a better way to code, which will hopefully result in less objects in memory, meaning you can delete them when you've finished with them.

[eluser]TheFuzzy0ne[/eluser]
My proposed solution is that you integrate some form of paging system. Pages can be limited to 500 results or so, and the client can get the first page, and the next, until there are no more results.

I know this probably isn't the solution you were looking for, but it is a scalable solution, and can potentially cope with as much data as you need to transfer.

[eluser]ArcticZero[/eluser]
Hmm, I was afraid that was the case. But anyway, I'm relieved this problem is finally getting some direction. I suppose I could try sending a separate call for every 100 rows of sales. So theoretically, I would only need to redo the main loop in the client side, right?

I really appreciate all the work you've done so far. Thank you.

[eluser]TheFuzzy0ne[/eluser]
Exactly so. Setting a hard limit on the results just makes your solution scalable, and it won't break if there too many results. You just need to bear in mind when setting a hard limit, that your data is variable, so you don't want a string that's longer than most breaking your application. For me, with my memory usage capped at 128MB, the application maxed out the memory limit on the 727th iteration of the main loop, so I would suggest going with something like 250 results per page, or at a push, 500 (if you're sure that'll work), and maybe raise the memory limit to 256M to be on the safe side, if you are able to do that. If you are on a shared host, you might want to let them know as they might not want you to.