3

I have a small script that updates a database. It takes about a minute or so to process currently, but the time will go up as the database goes up. I want to provide updates to the user as the script runs so that they know that the status of the script and that it is still going. I implemented Server-Side-Events and while it works, none of the updates are posted to the website until the script is completely finished.

I can upload the entire script file, but here are the relevant parts.

First, at the start of the script I check to see if the script is run from the command line and if not I send the headers as shown:

$cli = (PHP_SAPI === 'cli') ? 1 : 0;
if(!$cli){
    header( 'Content-Type: text/event-stream' );
    header( 'Cache-Control: no-cache' );
}

Here is my function to send an sse_message when called

function sse_message( $type, $message, $progress=0 )
{
    $d = array("type" => $type, "msg" => $message , "progress" => $progress);
    $id = time();

    echo "id: $id" . PHP_EOL;
    echo "data: " . json_encode($d) . PHP_EOL;
    echo PHP_EOL;

    ob_flush();
    flush();
}

And finally here is my javascript implementation of the listener:

if( typeof(EventSource) !== "undefined" ){
    source = new EventSource( "../priv3/script/prowip_process.php" );
    source.addEventListener("message", function(e)
    {
        var result = JSON.parse(e.data);

        switch(result.type){
            case "update":
                if( result.msg != "" )
                    document.getElementById("prowip_status").value += "\n[INFO]: "+result.msg;
                document.getElementById("prowip_progress").value = result.progress;
                break;

            case "success":
                document.getElementById("prowip_status").value += "\n[COMPLETED]:"+result.msg;
                document.getElementById("prowip_progress").value = 100;
                document.getElementById("button_prowip_exit").disabled = false;
                source.close();
                break;

            case "error":
                document.getElementById("prowip_status").value += "\n[ERROR]: "+result.msg;
                document.getElementById("button_prowip_exit").disabled = false;
                source.close();
                break;      
        }
    }, false);
}
else{
    document.getElementById("prowip_status").value += "\n\n[ERROR]: Your browser does not support server-sent events.  Please upgrade your browser or use a different browser for this function.  Please note this script will not work with Internet Explorer."
    document.getElementById("button_prowip_exit").disabled = false;
}

Again - everything works perfect except that no updates occur during the script process and then when it finishes all of the messages and progress bar is updated.

I have tried this in several browsers, computers, etc. but none of the updates to the webpage occur until the entire script is done.

Any help or comments would be greatly appreciated. Thanks.

EDITED 4/3/2015 TO POST MORE CODE AS REQUESTED

These 3 functions call sse_message or just echo for CLI.

function show_error( $cli, $message )
{
    if ($cli)
        echo $message."\n";
    else
        sse_message( "error", $message );
}

function show_message( $cli, $message, $progress )
{
    if ($cli)
        echo $message."[".$progress."%]\n";
    else
        sse_message( "update", $message, $progress );
}

function show_success( $cli, $message )
{
    if ($cli)
        echo $message."[100%]\n";
    else
        sse_message( "success", $message );
}

Here is a clip out of the main function that shows how these functions are called:

foreach ( $projects as $number => $project ) {
            $cn = strtok($number, "." );
            $pn = strtok(".");
            $wipover = $project['wip'] - $project['wip30'] - $project['wip60'] - $project['wip90'];
            $arover = $project['ar'] - $project['ar30'] - $project['ar60'] - $project['ar90'];
            $process_counter++;
            if( $process_counter >= $check ){
                $progress+=5;
                $check += intval ( $counter / 16 );
                show_message($cli,"",$progress);
            }
            if( ($project['empty']==1) and ($project['wip']==0) and ($project['ar']==0) ){
                //Do nothing...
            }
            else{
                $write_counter++;
                $update_query = "update projects set wip='{$project['wip']}', wip30='{$project['wip30']}', wip60='{$project['wip60']}', wip90='{$project['wip90']}', wipover='$wipover', ar='{$project['ar']}', ar30='{$project['ar30']}', ar60='{$project['ar60']}', ar90='{$project['ar90']}', arover='$arover' where client_num='$cn' and new_num='$pn'";
                if( ! $db->query( $update_query ) ){
                    show_error( $cli, "ERROR - update query: [$update_query] returned error: [".$db->error."]", $progress );
                }
            }
        }
Big D
  • 81
  • 1
  • 5
  • Can you show the main loop (i.e. the part of the PHP script that calls `sse_message`) ? If you set your CLI version to also call `sse_message()`, how many times does it get called during your minute's processing, when run from the commandline? – Darren Cook Apr 04 '15 at 20:04
  • I have 3 simple functions that are called depending upon the type of message, so the CLI version does not call sse_message and instead just uses a screen echo. I posted those functions to the original message. The main loop then calls one of these 3 functions. I have posted a clip of that process. I can post the entire function if that would be helpful. The sse_message is probably called about 30 times right now. – Big D Apr 05 '15 at 10:20
  • It all seems reasonable, doesn't it. If you look in the developer console of your browser, the network tab, do you see the received bytes are gradually going up. Or do you see 0 bytes received, then all the bytes are received at the end? My guess is the latter, and that the caching is happening server-side, (or at an intermediate caching or proxy server). You are using Apache, and the normal php apache module? – Darren Cook Apr 06 '15 at 08:40
  • Thanks Darren. You are correct about all the bytes being received at the end of the script. I am running this off of a fairly recent (<1 month) install of a virtual server with fairly stock configuration files. It is a basic LAMP setup on Ubuntu Linux. I do think you have isolated the problem. I will look further into the caching and see if I can find anything. I had hoped that passing the no-cache in the headers would take care of this. Thanks a ton for your comments and responses. Despite not being resolved, I at least have a direction to look! – Big D Apr 06 '15 at 16:52
  • Did you solve it? I have the same problem with this difference that my event stream worked and then suddenly stopped. Now data is showed only when script stopped. – Makla Jun 03 '15 at 11:09

1 Answers1

2

I had the same issue. It turned out to be the web server caching the output from php.

Within PHP ob_flush() and flush() forces the output from PHP to the web server, but cannot force the web server to push it out to the client.

My solution was to pad the output to 800,000 characters before flushing from PHP. i.e:

echo "data: " . str_pad($message, 800000) . PHP_EOL . PHP_EOL;

Each response back to the client is approx 6Kb big, so it's not very efficient but it was the only way I could get it to work.

Hope that helps!

duster
  • 21
  • 2