php and ajax: show progress for long script

Question

I have php script which can take quite a lot of time (up to 3-5 minutes), so I would like to notify user how is it going.

I read this question and decided to use session for keeping information about work progress.

So, I have the following instructions in php:

public function longScript()
{
       $generatingProgressSession = new Zend_Session_Namespace('generating_progress');
       $generatingProgressSession->unsetAll();

       .... 

       $generatingProgressSession->total = $productsNumber;

       ...

        $processedProducts = 0;
        foreach($models as $model){

             //Do some processing
             $processedProducts++;
             $generatingProgressSession->processed = $processedProducts;

    }

}

And I have simple script for taking data from session (number of total and processed items) which return them in json format.

So, here is js code for calling long script:

$.ajax({
    url: 'pathToLongScript',
    data: {fileId: fileId, format: 'json'},
    dataType: 'json',
    success: function(data){
        if(data.success){
            if(typeof successCallback == "function")
                successCallback(data);
        }

    }
});

//Start checking progress functionality
var checkingGenerationProgress = setInterval(function(){
$.ajax({
 url: 'pathToCheckingStatusFunction',
 data: {format: 'json'},
 success: function(data){
      console.log("Processed "+data.processed+" items of "+data.total);
      if(data.processed == data.total){
          clearInterval(checkingGenerationProgress);
      }
   }
  });
 }, 10000)

So, long scripted is called via ajax. Then after 10 seconds checking script is called one time, after 20 second - second time etc.

The problem is that none of requests to checking script is completed until main long script is complete. So, what does it mean? That long script consumes too many resources and server can not process any other request? Or I have some wrong ajax parameters?

See image:

-----------UPD

Here is a php function for checking status:

public function checkGenerationProgressAction()
{
    $generatingProgressSession = new Zend_Session_Namespace('generating_progress');
    $this->view->total = $generatingProgressSession->total;
    $this->view->processed = $generatingProgressSession->processed;
}

I'm using ZF1 ActionContext helper here, so result of this function is json object {'total':'somevalue','processed':'another value'}

Answer 1

I'd

exec ('nohup php ...');

the file and send it to background. You can set points the long running script is inserting a single value in DB to show it's progress. Now you can go and check every ten or whatever seconds if a new value has been added and inform the user. Even might be possible to inform the user when he is on another page within your project, depending on your environment.

Answer 2

Yes, it's possible that the long scripts hogs the entire server and any other requests made in that time are waiting to get their turn. Also i would recommend you to not run the check script every 10 seconds no matter if the previous check has finished or not but instead let the check script trigger itself after it has been completed.

Taking for example your image with the requests pending, instead of having 3 checking request running at the same time you can chain them so that at any one time only one checking request is run.

You can do this by replacing your setInterval() function with a setTimeout() function and re-initialize the setTimeout() after the AJAX check request is completed

Answer 3

Can you paste the PHP of "pathToCheckingStatusFunction" here?

Also, I notice that the "pathToCheckingStatusFunction" ajax function doesn't have a dataType: "json". This could be causing a problem. Are you using the $_POST['format'] anywhere?

I also recommend chaining the checks into after the first check has completed. If you need help with that, I can post a solution.

Edit, add possible solution:

I'm not sure that using Zend_namespace is the right approach. I would recommend using session_start() and session_name(). Call the variables out of $_SESSION.

Example File 1:

session_name('test');
session_start();
$_SESSION['percent'] = 0;

...stuff...

$_SESSION['percent'] = 90;

Example File 2(get percent):

session_name('test');
session_start();
echo $_SESSION['percent'];

Answer 4

Most likely, the following calls are not completing due to session locking. When one thread has a session file open, no other PHP threads can open that same file, as it is read/write locked until the previous thread lets go of it.

Either that, or your Server OR Browser is limiting concurrent requests, and therefore waiting for this one to complete.

My solution would be to either fork or break the long-running script off somehow. Perhaps a call to exec to another script with the requisite parameters, or any way you think would work. Break the long-running script into a separate thread and return from the current one, notifying the user that the execution has begun.

The second part would be to log the progress of the script somewhere. A database, Memcache, or a file would work. Simply set a value in a pre-determined location that the follow-up calls can check on.

Not that "pre-determined" should not be the same for everyone. It should be a location that only the user's session and the worker know.