ffmpeg video conversion

[eluser]montassis[/eluser]
Hey All,

I am writing a CI application where users can upload video files. I have read quite a lot about doing video conversion using ffmpeg and understand the basics. My problem is that I cannot seem to work out how to run this process in the background so the website doesn't just hang on a blank page while the video is being converted. Currently, I am attempting to run ffmpeg as a background process, but the view is not loaded until the conversion is complete.

Here is a portion of code from my Upload controller

Code:

function uploadVideo() {
        $config['upload_path'] = getcwd() . '/uservids/' . $this->session->userdata('accountid');
        $config['allowed_types'] = 'mov|wmv|ogv';
        $filename = md5(date(DATE_RFC822));
        $config['file_name'] = $filename;

        $this->load->library('upload', $config);

        if (!$this->upload->do_upload('videoFile')) {
            [removed code]
        } else {

            $videoDetails = $this->upload->data();
            $videoDetails['accountid'] = $this->session->userdata('accountid');
            $videoid = $this->_createVideo($videoDetails);

            // page data
            $mainData['success'] = $this->upload->data();
            $mainData['firstName'] = $this->session->userdata('firstName');
            $mainData['lastName'] = $this->session->userdata('lastName');

            // form data
            $mainData['title'] = array('name' => 'title', 'id' => 'title');
            $mainData['comment'] = array('name' => 'title', 'id' => 'title', 'rows' => '8', 'cols' => '37');
            $mainData['videoid'] = array('name' => 'videoid', 'value' => $videoid);

            // load the view
            $data['mainView'] = 'upload/upload_success';
            $data['mainData'] = $mainData;
            $this->load->view('template', $data);

            // conversion
            $command = 'ffmpeg -i ' . $videoDetails['full_path'] . ' ' . $videoDetails['file_path'] . $filename . '.ogv';

            echo("Running file conversion");
            $ps = $this->_runInBackground($command);
            // while process is running, print a nice little dot
            while ($this->_isProcessRunning($ps)) {
                echo(" . ");
                ob_flush();
                flush();
                // sleep for .25 seconds
                usleep(250000);
            }
                
        }
    }

    // inserts into database, returns new videoid
    function _createVideo($details) {
        return $this->Video->createVideo($details);
    }

    // executes function to run in the background (not working)
    function _runInBackground($command, $priority = 0) {
        if ($priority)
            $PID = shell_exec("nohup nice -n $priority $command 2> /dev/null & echo $! &");

        else
            $PID = shell_exec("nohup $command > /dev/null 2> /dev/null & echo $! &");
        return($PID);
    }

    // this works
    function _isProcessRunning($PID) {
        exec("ps $PID", $ProcessState);
        return(count($ProcessState) >= 2);
    }
}

I know the conversion is working, because I get the output file and a line up the top that says "Running file conversion................................" with a bunch of dots.

So does anyone know how to run this type of thing in the background? Also, if possible, rather than sending the output to /dev/null I wouldnt mind sending it to a text file that I could read and then get info like the duration and number of frames so i could come up with a progress bar.

Any help or info much appreciated.

[eluser]Pascal Kriete[/eluser]
I'm pretty sure your browser is buffering the response until it has a dom structure that makes sense. If you open a connection with something like telnet you should see the dots coming in.

It might be simpler to just load up the page and store the PID away somewhere. Then set up an ajax polling function that checks progress every few seconds.

I'm very opposed to premature optimization, but if you expect more than a handful of people to upload videos, I would definitely consider installing gearman (or something similar). Web servers need cpu, databases and caches need ram. Unfortunately converting video also needs cpu. So even just three or four conversions will bring that frontend to a crawl. You'll want the process on a different box.

[eluser]Unknown[/eluser]
The way i handled this was to write a conversion queue which included a table, some php scripts and cron.

[eluser]montassis[/eluser]
Hey thanks for the tips. I have looked into gearman and its definitely something I will probably implement down the track, at the moment I just want to get things working to make sure what I'm doing is feasible.

I tried to set it up to poll every few seconds using ajax but unfortunately the process still doesnt load in the background... so the view loads, the video starts getting converted but then waits until its finished and then the polling starts.

I'd rather avoid doing crons, ultimately i'd like a progress bar to show progress (which I can create using ffmpeg output). The videos arent very large and the max time allowable is 30 seconds (which i need to determine as the video is uploaded using ffmpeg, so crons probably arent going to cut it there either).

Here is my updated code. I have now set the PID to a hidden field, which is read by the doc ready function in the javascript (jquery plugin periodicalupdater) and polls the new method videoProgress(). As I said though, it wait until the ffmpeg has finished, and prints a whole bunch of dots.

Upload Controller

Code:

function uploadVideo() {
        $config['upload_path'] = getcwd() . '/uservids/' . $this->session->userdata('accountid');
        $config['allowed_types'] = 'mov|txt|pdf|doc|wmv|ogv';
        $filename = md5(date(DATE_RFC822));
        $config['file_name'] = $filename;

        $this->load->library('upload', $config);

        if (!$this->upload->do_upload('videoFile')) {
            [removed]
        } else {

            $videoDetails = $this->upload->data();
            $videoDetails['accountid'] = $this->session->userdata('accountid');
            $videoid = $this->_createVideo($videoDetails);

            $fullpath = $videoDetails['full_path'];
            $filepath = $videoDetails['file_path'];

            // conversion
            $command = 'ffmpeg -i ' . $fullpath . ' ' . $filepath . $filename . '.ogv';

            $pid = $this->_runInBackground($command);

            // page data
            $mainData['success'] = $this->upload->data();
            $mainData['firstName'] = $this->session->userdata('firstName');
            $mainData['lastName'] = $this->session->userdata('lastName');

            // form data
            $mainData['title'] = array('name' => 'title', 'id' => 'title');
            $mainData['comment'] = array('name' => 'title', 'id' => 'title', 'rows' => '8', 'cols' => '37');
            $mainData['hidden'] = array(
                'videoid' => $videoid,
                'pid' => $pid);

            //load scripts
            $data['scripts'] = array(
                '/mogo/js/periodical/periodicalupdater.js',
                '/mogo/js/upload_success.js');

            // load the view
            $data['mainView'] = 'upload/upload_success';
            $data['mainData'] = $mainData;
            $this->load->view('template', $data);
        }
    }

    //inserts into database, returns id
    function _createVideo($details) {
        return $this->Video->createVideo($details);
    }

    // supposed to run a command in the background
    function _runInBackground($command, $priority = 0) {
        if ($priority)
            $PID = shell_exec("nohup nice -n $priority $command 2> /dev/null & echo $! &");

        else
            $PID = shell_exec("nohup $command > /dev/null 2> /dev/null & echo $! &");
        return($PID);
    }

    // checks if process is running
    function _isProcessRunning($PID) {
        exec("ps $PID", $ProcessState);
        return(count($ProcessState) >= 2);
    }

    // used by jquery call to check progress
    function videoProgress() {

        $pid = $this->input->post('pid');

        while ($this->_isProcessRunning($pid)) {
            echo ' . ';
            ob_flush();
            flush();
            // sleep for .25 seconds
            usleep(250000);
        }
    }

Javascript

Code:

$().ready(function() {
    
    var pid = $('[name|="pid"]').val();
    
    $.PeriodicalUpdater('videoProgress', {
        method: 'post',          // method; get or post
        data: {pid: pid},                   // array of values to be passed to the page - e.g. {name: "John", greeting: "hello"}
        minTimeout: 250,       // starting value for the timeout in milliseconds
        maxTimeout: 2000,       // maximum length of time between requests
        multiplier: 2,          // if set to 2, timerInterval will double each time the response hasn't changed (up to maxTimeout)
        type: 'text',           // response type - text, xml, json, etc.  See $.ajax config options
        maxCalls: 0,            // maximum number of calls. 0 = no limit.
        autoStop: 0             // automatically stop requests after this many returns of the same data. 0 = disabled.
    }, function(data) {
        $('#progress').append(data);
    });
});

Any suggestions?

[eluser]montassis[/eluser]
After reading the code I posted I think the while loop in videoProgress() should be changed to an if, will try that.

[eluser]Pascal Kriete[/eluser]

After reading the code I posted I think the while loop in videoProgress() should be changed to an if, will try that.

Yep, that's what I was thinking - check for completion once per ajax request rather than trying to stream the results.