Hi all- I've tested this as thoroughly as I can and continue to be baffled where the buildup is coming from. If anyone wants to recreate or make suggestions please do.
Note: I have already worked around this, but would like to determine if this is a framework issue so it gets addressed.
I am using a custom CLI command (App\Commands) to import a large CSV into the database. The actual processing is very simple, and uses existing models (with internal commands only, no modification) to save to the database. After processing a reasonable ~200k lines the script crashes with "Allowed memory size ... exhausted". However, there should be no accruing memory storage as all the variables are reused on each iteration. I suspect something in the model is hanging onto a variable by reference such that it doesn't get cleared on each iteration.
Here's a generalized (but still accurate) version of the code:
PHP Code:
public function run(array $params)
{
$myModel = new MyModel();
$myModel2 = new MyModel2();
$file = "data.csv";
$row = 0;
if (($handle = fopen($file, "r")) !== false):
while (($data = fgetcsv($handle, 1000, ",")) !== false):
$row++;
/* VALIDATE DATA */
// validate line
$num = count($data);
if ($num != 4):
CLI::write("Invalid number of columns on line {$row}: ". CLI::color(implode(",", $data), 'yellow'));
return;
endif;
// get the timestamp
if ($timestamp = strtotime($data[2])):
else:
CLI::write("Invalid timestamp on line {$row}: ". CLI::color($data[2], 'yellow'));
return;
endif;
/* PRIMARY ENTRY */
// build the row
$row = [
'source' => "import",
'created_at' => date("Y-m-d H:i:s", $timestamp)
];
// add to the database
$myModel->save($row);
$ref_id = $myModel->getInsertID();
/* DEPENDENT ENTRY */
// build the row
$row2 = [
'ref_id' => $ref_id,
'content' => $data[1],
];
$myModel2->save($row2);
endwhile;
fclose($handle);
endif;
}
