XML-RPC memory leak question..

[eluser]ArcticZero[/eluser]
I currently have an XML-RPC server set up using CI on Windows. The client gathers sales data and sends it to the server, where it is stored in a centralized database. Although when large XML files are sent, I keep running out of memory.

Code:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 1800 bytes)

I noticed that whenever the processing moves to the server (after the client finishes preparing it), Apache's memory usage just shoots up continuously until it hits the limit. I take it it's using quite a great deal of memory. I've even tried disabling all server-side processing on my part, and sending a canned response instead regardless of the data. But despite this, the memory leak persists.

Using Xdebug, I see it says that it's CI's XMLRPC library that takes up a lot of memory. What gives? I'm totally clueless at the moment as to what to do, and I need to get this project done ASAP.

I'm curious as to whether CI "stores" something much like how it stores SQL queries by default (which I also have disabled).

[eluser]TheFuzzy0ne[/eluser]
I'm sorry to hear you're having trouble. Personally, I don't use the library, but it is certainly sounding like it might be a bug.

It would be great if you could find the time to create a test of some kind that would duplicate your problem, so that others can test it and hopefully find a solution. As CodeIgniter is community driven, feedback from the community is vital. I know this wasn't the answer you were looking for, but it really help if we could replicate your problem.

[eluser]ArcticZero[/eluser]
I suppose I can create a test script which generates bogus data. I mean, I'm under the assumption that if you try sending a large mass of data to an XML-RPC server on CI, you simply have to check your resource usage, and watch Apache's (httpd.exe) memory counter go up.

EDIT: I might as well post my code for both the client and server. I'm including the XML-RPC library I used for the client as well.

Client Files

Server (CI controller)

I'm desperately hoping for any assistance on this. And I do apologize I couldn't just paste the code here. Not enough characters per post to accomodate it. Thanks!

[eluser]TheFuzzy0ne[/eluser]
Thanks. I'll make time to look at the code as soon as I can, then I'll get back to you.

[eluser]TheFuzzy0ne[/eluser]
The only thing I'm missing now is some data to submit, assuming that the problem is when you use submit_transactions.

[eluser]ArcticZero[/eluser]

Code:

CREATE TABLE `sales` (
  `date` date NOT NULL default '0000-00-00',
  `customer_id` int(8) NOT NULL default '0',
  `sale_sub_total` varchar(12) NOT NULL default '',
  `sale_total_cost` varchar(30) NOT NULL default '',
  `paid_with` varchar(25) NOT NULL default '',
  `items_purchased` int(8) NOT NULL default '0',
  `percent_discount` varchar(5) NOT NULL default '0',
  `sold_by` int(8) NOT NULL default '0',
  `comment` varchar(100) NOT NULL default '',
  `id` int(8) NOT NULL auto_increment,
  `amount_tendered` varchar(15) default NULL,
  `change` varchar(15) default NULL,
  `status` int(1) NOT NULL default '0',
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 COMMENT='Contains overall sale details' AUTO_INCREMENT=11 ;

CREATE TABLE `sales_items` (
  `sale_id` int(8) NOT NULL default '0',
  `item_id` int(8) NOT NULL default '0',
  `quantity_purchased` int(8) NOT NULL default '0',
  `item_unit_price` varchar(15) NOT NULL default '0',
  `item_buy_price` varchar(30) NOT NULL default '0',
  `item_tax_percent` varchar(10) NOT NULL default '0',
  `item_total_tax` varchar(12) NOT NULL default '0',
  `item_total_cost` varchar(12) NOT NULL default '0',
  `id` int(8) NOT NULL auto_increment,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM  DEFAULT CHARSET=latin1 COMMENT='Table that holds item information for sales' AUTO_INCREMENT=1 ;

Just did an export of the sales/sales_items table structures for reference. I do appreciate the time you're putting into this.

Also, here is a sample script I use to insert lots of data for testing purposes:

Code:

<?php
    $sql = mysql_connect("localhost", "root", "");
    $db = mysql_select_db("clientdatabase", $sql);
    
    $sales = 10000;
    $items_per_sale = 10;
    
    for($i = 0; $i < $sales; $i++)
    {
        $last_id = false;
        $query = mysql_query("INSERT INTO sales (`date`, customer_id, sale_sub_total, sale_total_cost, paid_with, items_purchased, percent_discount, sold_by, amount_tendered, `change`, `status`) VALUES ('2008-02-19', 7, '4000.50', '4000.50', 'Cash', '20', '0', 1, '4001', '0.50', 0)", $sql);
        
        $last_id = mysql_insert_id($sql);
        
        for($z = 0; $z < $items_per_sale; $z++)
        {
            $query = mysql_query("INSERT INTO sales_items (sale_id, item_id, quantity_purchased, item_unit_price, item_buy_price, item_tax_percent, item_total_tax, item_total_cost) VALUES (" . $last_id . ", 1, 5, 110, 100, 10, 10, 550)", $sql);
        }
    }
    
    if($query)
    {
        echo "$sales records inserted.";
    }
    else
    {
        echo mysql_error($sql);
    }
?&gt;

Again, thank you very much!

[eluser]TheFuzzy0ne[/eluser]
OK, I'm still having a look through the code, I just thought I should point out that you've published your database password for root.

Can you show me what query you fire at the controller that results in the memory limit being exceeded? I'm not very familiar with RPC, so it would really save me a lot of reading.

[eluser]ArcticZero[/eluser]
I've taken the password out prior to posting, however even so it's perfectly fine. It's a development machine anyway.

Any one of these could be the actual message sent, depending on the conditions met:

Code:

if(isset($_GET['test']))
{
    $msg = new xmlrpcmsg('Submit_transactions', array(new xmlrpcval('test', 'string')));
}
else
{
    if($transactions && $depleted)
    {
        $msg = new xmlrpcmsg('Submit_transactions', array(new xmlrpcval($final_array, 'struct'), new xmlrpcval($my_hash, 'string'), new xmlrpcval($version, 'string'), new xmlrpcval($depleted_array, 'array')));
    }
    else if($transactions && !$depleted)
    {
        $msg = new xmlrpcmsg('Submit_transactions', array(new xmlrpcval($final_array, 'struct'), new xmlrpcval($my_hash, 'string'), new xmlrpcval($version, 'string'), new xmlrpcval('None', 'string')));
    }
    else if(!$transactions && $depleted)
    {
        $msg = new xmlrpcmsg('Submit_transactions', array(new xmlrpcval('None', 'string'), new xmlrpcval($my_hash, 'string'), new xmlrpcval($version, 'string'), new xmlrpcval($depleted_array, 'array')));
    }
    else
    {
        $msg = new xmlrpcmsg('Submit_transactions', array(new xmlrpcval('None', 'string'), new xmlrpcval($my_hash, 'string'), new xmlrpcval($version, 'string'), new xmlrpcval('None', 'string')));
    }
}

And this is the command that sends it to the server:

Code:

$response = $client->send($msg);

And this parses the server's response:

Code:

$res = php_xmlrpc_decode($response->value());

I apologize if I can't explain it any clearer, as this is my first attempt at XML-RPC as well. The whole thing works with less rows to retrieve. Thanks!

[eluser]TheFuzzy0ne[/eluser]
What version of the adodb abstraction layer are you using?

[eluser]TheFuzzy0ne[/eluser]
OK, I just grabbed the latest version that works with both PHP 4 and 5. Now I'm getting this error:

Fatal error: Allowed memory size of 67108864 bytes exhausted (tried to allocate 3145728 bytes)

Now it's debugging time.