Tracking long running php scripts externally

[eluser]CroNiX[/eluser]
First I'll explain what I'm trying to do as it seems kind of complex. I haven't had the need to do something like this before so it's new territory.

I have some CI scripts that take a long time to process (minutes to hours). Some are scheduled and run via cron, which works fine and send out email reports upon completion.

However, some of these I need/want to trigger manually at will, via the browser, like a normal CI page but I do not want to wait for the job to complete before rendering output to the browser (or even bypass CI's buffer and stream realtime, which I also do). Basically log into a control panel and be able to click a link or something to trigger a job and exit without waiting for the actual job to start/finish.

What I was thinking is that when this "job controller" is called to perform a function, by clicking on its link, is that it would get some sort of unique ID for this "process" which it can send back to the browser AND also pass this ID to the "process" so that that process can use the ID to log it's activity into a database using that ID, at which time the main script would exit/terminate connection with the browser as soon as it receives the ID and the other main process runs on it's own in the background without having to have the browser connection "open". If the ID is known, a separate script could then poll the database at regular intervals (if wanted) to get the output of the process from the database by it's unique ID that was passed back from the browser. I was thinking of also having a jobs db table which would track which jobs were currently running so I can also find that out.

From some reading, it seems this is possible by using pcntl_fork(), however I am not easily understanding this.

If this is the right direction, how do I:
1) Get ID of new child process that I would immediately pass back to the browser (this will most likely be via ajax/json and once the ID is returned back by the ajax use a separate ajax call on a timer to retrieve the status of that process)
2) Have that child process know that ID, or be able to pass that ID to it

If anybody has any ideas or have done something similar, I'd love to hear about it. Also if there are any pitfalls to using CI to do this.

Thanks!

[eluser]PhilTem[/eluser]
I think I've done something similar just recently when I was setting up my server to use the GitHub POST-commit hook and do a dirty deployment of my page. The first two lines of my deployment script are the following

Code:

// This will continue script execution even if the page was closed in the browser
//  or the request ended on a different way
ignore_user_abort(TRUE);

// Important to set the time limit otherwise you won't be able to deploy huge
//  commits/pushes since the script will just timeout
set_time_limit(30);

Actually it's more than two lines but there are just two lines really doing something. I think in your case you might wanna check out the PHP docu on ignore_user_abort since this should basically - at least I hope so - solve your problem: Scripts won't stop executing once the URL was called and the browser window closed while loading.
Combined with some AJAX this could work to have a link change from something like "start" to "running..."

[eluser]CroNiX[/eluser]
That's part of it, but I was really wanting to spawn a new process (fork) so this script can process several things at once in parallel instead of sequentially. It also doesn't send any output to the user until the entire script finishes. What it does do is allow the user to navigate away from the page and have the script finish in the background.

This gets closer (sends output immediately) but still processes sequentially.

Code:

//Unlimited time and ignore user abort
ignore_user_abort(TRUE);
set_time_limit(0);

//capture the output
ob_start();
echo json_encode($data);

//Send json header along with no-cache
header('Cache-Control: no-cache, must-revalidate');
header('Expires: Mon, 26 Jul 1997 05:00:00 GMT');
header('Content-type: application/json');
  
//Get length of (json) output and terminate connection to browser to allow background process to continue
header("Content-Length: " . ob_get_length());
header('Connection: close');
  
//Send it all out, we're done
ob_end_flush();
ob_flush();
flush();

//Now browser has the output and script can continue...
//rest of long running code here...