allow PHP script with long execution time to send updates back to the browser

Question

I looked over a few of the questions, namely

and neither one seems to answer my question, part of which seems to be "how do I do this?" and the other half is "Hey, the way I'm doing this right now - is it the best way? Could I code this better?"

I have a simple ajax script that sends some data over to a PHP script:

$.ajax({
    type: 'POST',
    url: 'analysis.php',      
    data: { reportID:reportID, type:type, value:value, filter_type:filter_type, filter_value:filter_value, year:year },        
    success:function(dataReturn){          
      analysis_data = JSON.parse(dataReturn);
      /* do stuff with analysis_data... */
});

This PHP script takes about 3 minutes to run, as it loops through a database and runs some pretty complex queries:

<?php
session_start();
ob_start();
ini_set('max_execution_time', 180);

$breaks = [ 1000, 2000, 4000, 6000, 8000, 10000, 20000, 50000, 99999999 ];
$breaks_length = count($breaks);
$p = 0;

foreach ( $breaks as $b ) {

  $p++;
  $percentage_complete = number_format($p / $breaks_length,2) . "%";

  $sql = "query that takes about 20 seconds to run each loop of $b....";  

  $query = odbc_exec($conn, $sql);
  while(odbc_fetch_row($query)){   
    $count = odbc_result($query, 'count');
  }

  $w[]  = $count;

  /* tried this... doesn't work as it screws up the AJAX handler success which expects JSON
  echo $percentage_complete;
  ob_end_flush();
  */
}

echo json_encode($w);
?>

All of this works - but what I'd really like to do is find a way after each foreach loop, to output $percentage_complete back to the user so they can see it working, instead of just sitting there for 2 minutes with a FontAwesome icon spinning in front of them. I tried using ob_start();, but not only does it not output anything until the page is done running, it echoes the value, which is then part of what is sent back to my AJAX success handler, causing it to screw up. (I need the output in a JSON_encoded format as I use it for something else later.)

So far in threads I've read, my only thought is to start the $breaks array loop on the previous page, so instead of looping 6 times on the same page, I loop once, return an answer, then call analysis.php again using the second element of the $breaks array, but I'm not sure this is the best way to go about things.

Also - during the 3 minutes that the user is waiting for this script to execute, they cannot do anything else on the page, so they just have to sit and wait. I'm sure there's a way to get this script to execute in such a way it doesn't "lock down" the rest of the server for the user, but everything I've searched for in Google doesn't give me a good answer for this as I'm not sure exactly what to search for...

You did json_encode($percentage_complete), right? You have nothing else in the JS callback except "do_stuff"? — Kevin_Kinsey, Aug 11 '16 at 19:52
Typically the callback would parse whatever is handed back to it and replace the innerHTML of some element on the page where you wanted the feedback to be displayed.... — Kevin_Kinsey, Aug 11 '16 at 19:53
The question isn't about how to handle the actual json response that the PHP page is giving back to the success call - it's a long section of code that I run in the success call to handle it, and it works fine. The question is whether or not there is any way for me, while the script is in the process of executing, to return something to the user so they can see progress happening in "real time" - and @Kkinsey, I tried outputting it as json - and while I could get it to work, it didn't output anything until the script had completely finished (which is meaningless to me...) — Brian Powell, Aug 11 '16 at 19:56
Does this help? http://www.htmlgoodies.com/beyond/php/show-progress-report-for-long-running-php-scripts.html — Kevin_Kinsey, Aug 11 '16 at 20:04
Alternatively looking for `readyState==3` might do the trick: http://stratosprovatopoulos.com/web-development/php/ajax-progress-php-script-without-polling/ — hsan, Aug 11 '16 at 20:14

score 1 · Answer 1 · answered Aug 11 '16 at 20:12

I have done this a couple different ways, but the pattern I like the best is to have three scripts (or one controller to handle all of this), analysis_create.php, analysis.php, and analysis_status.php. The key is to create a DB object that you reference in your status checks (analysis_status.php). analysis_create.php will store all the data in the post into a DB table that will also have a column for percent_complete. The analysis_create.php function should return an ID/Token for the analysis. Once the front-end has the ID, it would post to analysis.php and then after a delay (250ms) kill the request, because you don't want to wait for it to finish. analysis.php should read the data out of the DB and start doing the work. You will need to make sure ignore_user_abort is set properly in your analysis.php script. Once the request to analysis.php is killed, you will start long polling to analysis_status.php with that ID. As analysis.php is working through the query, it should be updating the corresponding DB record with the percentage complete. analysis_status.php should look up this record and return the percentage complete to the front end.

MonkeyZeus · Accepted Answer · 2016-08-11T20:32:17.787

You are encountering what is know as Session Locking. So basically PHP will not accept another request with session_start() until the first request has finished.

The immediate fix to your issue is to remove session_start(); from line #1 completely because I can see that you do not need it.

Now, for your question about showing a percentage on-screen:

analysis.php (modified)

<?php
ob_start();
ini_set('max_execution_time', 180);

$breaks = [ 1000, 2000, 4000, 6000, 8000, 10000, 20000, 50000, 99999999 ];
$breaks_length = count($breaks);
$p = 0;

foreach ( $breaks as $b ) {

  $p++;

  session_start();
  $_SESSION['percentage_complete'] = number_format($p / $breaks_length,2) . "%";
  session_write_close();

  $sql = "query that takes about 20 seconds to run each loop of $b....";  

  $query = odbc_exec($conn, $sql);
  while(odbc_fetch_row($query)){   
    $count = odbc_result($query, 'count');
  }

  $w[]  = $count;

  /* tried this... doesn't work as it screws up the AJAX handler success which expects JSON
  echo $percentage_complete;
  ob_end_flush();
  */
}

echo json_encode($w);

check_analysis_status.php get your percentage with this file

<?php
session_start();
echo (isset($_SESSION['percentage_complete']) ? $_SESSION['percentage_complete'] : '0%');
session_write_close();

Once your AJAX makes a call to analysis.php then just call this piece of JS:

// every half second call check_analysis_status.php and get the percentage
var percentage_checker = setInterval(function(){
    $.ajax({
        url: 'check_analysis_status.php',
        success:function(percentage){
            $('#percentage_div').html(percentage);

            // Once we've hit 100% then we don't need this no more
            if(percentage === '100%'){
                clearInterval(percentage_checker);
            }
        }
    });
}, 500);

I'm definitely using session variables in here, but I can easily call those from the previous page and pass them in my ajax POST section. Thank you though for taking so much time to answer this question! I really appreciate it! Let me work through everything in here, there's a lot to read. — Brian Powell, Aug 11 '16 at 20:32

score 0 · Answer 3 · answered Mar 28 '22 at 05:02

I ran into the same issue. What caused it is different to what people are suggesting here.

Reason was gzip was enabled. Leading to a type of session locking even without an actual session.

Several ways to disable for one specific file: How to disable mod_deflate in apache2?

Put this in httpd.conf

SetEnvIfNoCase Request_URI getMyFile\.php$ no-gzip dont-vary

allow PHP script with long execution time to send updates back to the browser

3 Answers3