Server Sent Events With Codeigniter 4

I am trying to update the progress of a long running php script on frontend using server sent events.

But am only receiving first few eventstreams and thereafter there is error. I am Sharing code below;

Html View:

Code:

<a href="#" onclick="startScraping()" >START SCRAPING</a>
<a href="#" onclick="stopScraping()"  >STOP SCRAPING</a>

<script>
        var source = 'THE SOURCE';

        function startScraping()
        {
            if(typeof(EventSource) !== "undefined") {
                
                source = new EventSource('<?= route_to('start-scraping') ?>');
            
                source.addEventListener('message' , function(e)
                {
                    var result = JSON.parse( e.data );
                    
                    addLog(result.message);

                    var el = document.getElementById('progressBar');
                    
                    el.style.width = result.progress + '%';
                    el.setAttribute('aria-valuenow', result.progress);

                    if(e.data.search('TERMINATE') != -1)
                    {
                        addLog('Received TERMINATE closing');
                        source.close();
                    }
                });

                source.addEventListener('error' , function(e)
                {
                    addLog('eventSource: error occurred');

                    if (this.readyState == EventSource.CONNECTING) {
                        console.log('eventSource: reconnecting (readyState=${this.readyState})...');
                    } else {
                        console.log("eventSource: connection failed");
                    }
                    
                    source.close();
                });

            } else {
                document.getElementById("result").innerHTML = "Sorry, your browser does not support server-sent events...";
            }


        }

        function stopScraping()
        {
            source.close();
            addLog('Interrupted');
        }
        
        function addLog(message)
        {
            var r = document.getElementById('results');
            r.innerHTML += message + '<br>';
            r.scrollTop = r.scrollHeight;
        }
</script>

Home Controller:

PHP Code:

<?php namespace App\Controllers;

class Home extends BaseController
{
         public function __construct(){}

    /**
     * Function to start web scraping
     */
    public function startScraping()
    {
        header("Content-Type: text/event-stream");
        header("Cache-Control: no-cache");
        
        while (ob_get_level() > 0) {
                ob_end_flush();
        }
        
        ini_set('max_execution_time', 0);

        $serverTime = time();

            for ($i=1; $i < 10000 ; $i++) { 

                 $percent = intval(($i/10000) * 100);

                 sendMsg($serverTime, $i.' row(s) Processed.' , $percent); 
             sleep(1);
            
            }

        sendMsg($serverTime, 'TERMINATE',100);
        flush();
            exit;

    }
} 


Common.php file:

PHP Code:

function sendMsg($id, $message, $progress) 
{
    $d = array('message' => $message, 'progress' => $progress);

    echo "id:". $id . PHP_EOL;
    echo "data:". json_encode($d) . PHP_EOL;
    echo PHP_EOL;

    ob_end_flush();
    flush();
} 


Here this loop is not fully executed. I am only receiving first few rows as response.  There after am getting this error


id:1608816937
data:{"message":"1 row(s) Processed.","progress":0}

<br />
<b>Fatal error</b>:  Uncaught ErrorException: Cannot modify header information - headers already sent by (output started at /var/www/webScraper/app/Common.php:130) in /var/www/webScraper/system/Debug/Exceptions.php:164
Stack trace:
#0 [internal function]: CodeIgniter\Debug\Exceptions-&gt;errorHandler(2, 'Cannot modify h...', '/var/www/webScr...', 164, Array)
#1 /var/www/webScraper/system/Debug/Exceptions.php(164): header('HTTP/1.1 500 In...', true, 500)
#2 [internal function]: CodeIgniter\Debug\Exceptions-&gt;exceptionHandler(Object(ErrorException))
#3 {main}
  thrown in <b>/var/www/webScraper/system/Debug/Exceptions.php</b> on line <b>164</b><br />
<br />
<b>Fatal error</b>:  Uncaught ErrorException: Cannot modify header information - headers already sent by (output started at /var/www/webScraper/app/Common.php:130) in /var/www/webScraper/system/Debug/Exceptions.php:164
Stack trace:
#0 [internal function]: CodeIgniter\Debug\Exceptions-&gt;errorHandler(2, 'Cannot modify h...', '/var/www/webScr...', 164, Array)
#1 /var/www/webScraper/system/Debug/Exceptions.php(164): header('HTTP/1.1 500 In...', true, 500)
#2 /var/www/webScraper/system/Debug/Exceptions.php(224): CodeIgniter\Debug\Exceptions-&gt;exceptionHandler(Object(ErrorException))
#3 [internal function]: CodeIgniter\Debug\Exceptions-&gt;shutdownHandler()
#4 {main}
  thrown in <b>/var/www/webScraper/system/Debug/Exceptions.php</b> on line <b>164</b><br />


What am i doing wrong?

Sounds like your re-sending the headers again hard to tell by the code fragments.

Check your headers they should only be sent once.

What did you Try? What did you Get? What did you Expect?

Joined CodeIgniter Community 2009.  ( Skype: insitfx )

PHP Code:

headers already sent by (output started at /var/www/webScraper/app/Common.php:130) 


Often the error for header already sent means there’s an error message that was printed. 
What do you have on line 130?

CodeIgniter 4 tutorials (EN/FR) - https://includebeer.com
/*** NO support in private message - Use the forum! ***/

PHP Code:

headers already sent by (output started at /var/www/webScraper/app/Common.php:130) 


Often the error for header already sent means there’s an error message that was printed. 
What do you have on line 130?

echo "id:". $id . PHP_EOL;

this is the code i have on line 130 on common.php file.  complete function is as below

PHP Code:

function sendMsg($id, $message, $progress) 
{
    $d = array('message' => $message, 'progress' => $progress);

    echo "id:". $id . PHP_EOL;    // Line number 130
    echo "data:". json_encode($d) . PHP_EOL;
    echo PHP_EOL;
    
    //PUSH THE data out by all FORCE POSSIBLE
    ob_end_flush();
    flush();
}