Provide feedback from Python script to PHP

Question

Suppose a python script that performs a large number of operations, and outputs to the user a progress report while it is running:

$ script.py
Doing 1 of 100 operations...
Operation 1 succeeded with output "023948"
Doing 2 of 100 operations...
Operation 2 succeeded with output "893232"
Doing 3 of 100 operations...
Operation 3 succeeded with output "580217"
Doing 4 of 100 operations...
Operation 4 succeeded with output "228906"

Each line of output is shown about 2-3 seconds after the previous line, so the entire script run may take upwards 300 of seconds. I would like to run this script from a PHP-generated web page, and display the output to the user while it is running. I know that I can poll the PHP script from AJAX and get the latest message to update to the screen, but how can I get the actual output from the Python script while it is running? Functions such as exec() and shell_exec() only return the output after the script has finished running

In the worst case I could modify the Python script to output to a file, and to have an AJAX script continuously ping a PHP script which will read the file and diff it to the last read (also stored on the file system). But I would rather not modify the Python script for various reasons, and additionally I don't particularly like the idea of maintaining two additional files on the filesystem.

Did you try proc_open? http://php.net/manual/de/function.proc-open.php — ToBe, May 23 '13 at 14:41
possible duplicate of [How to send data using POST in python to php](http://stackoverflow.com/questions/4214231/how-to-send-data-using-post-in-python-to-php) — Ohgodwhy, May 23 '13 at 14:41
@ToBe: I am toying around with `proc_open()` but it does not seem to provide the functionality that I need, see my answer below. — dotancohen, May 23 '13 at 15:12
@Ohgodwhy: Thank you. This is not a dupe of the question that you posted, it is not even close other than the fact that the other question is also trying to tie PHP and Python together. What aspects do you feel are a dupe? — dotancohen, May 23 '13 at 15:13

Fernando Jorge Mota · Accepted Answer · 2013-05-24T00:45:47.143

Use unbuffered stdout, passing the argument -u to the python script at the opening of the daemon (something as python -u (your python script))

And, in PHP, use something such as proc_open to read the content printed by the Python Script in real time.

Edit

As specified in comments, i can suggest something as:

Python:

import sys, atexit
sys.stdout = open(sys.argv.pop(), "w+") #Replaces stdout with a file returned from sys.argv (command line arguments)
def saveClose():
    sys.stdout.write("--%s--"%sys.stdout.name) #Just to indicate if the script closed
atexit.register(saveClose) #Register with atexit to execute the function at...exit

PHP: (named as daemon.php)

<?php
function execInBackground($cmd) {  // Put the program in background in Windows and *nix
    if (substr(php_uname(), 0, 7) == "Windows"){ // Detect if Windows 
        pclose(popen("start /B ". $cmd, "r")); // Use start /B (windows only) to open a background program in Windows
    } 
    else { 
        exec($cmd . " > /dev/null &");  // Open program as a daemon using & in *nix.
    } 
} 
if(isset($_GET["verify_id"])){ // We have ID?
  $content = file_get_contents($_GET["verify_id"]); // If yes, just load the file here (this is a security problem, but you can fix easily)
  echo $content; // Simply echoes the content of the file
}
else if(isset($_GET["daemon"])){
  $id = md5(uniqid(rand(), true)); // Create a unique hash
  execInBackground($_GET["daemon"]." ".$id); // Execute in the background passing the hash as a argument
  echo $id; // Echoes the hash
}
?>

Javascript: (named as daemon.js and with use of jQuery)

var cmds = {}
function receiveResult(cmd, id, callback){ // This function effectively receives the result from the execution of the program.
   var reg = new RegExp("--"+id+"--$");
   cmds_interval[id] = setInterval(function(){
       $.ajax({
         url:"daemon.php",
         dataType: "text",
         data: {"verify_id":id},
         success: function(msg){
           if(reg.test(msg)){ // Is the script closed?
              msg = msg.replace(reg, ""); // If yes, removes it from the end of the string
              clearInterval(cmds_interval[id]); // And clear the interval
           }
           callback(msg, id, cmd); // Callback with the message from the stdout 
         }
      });
   }, 1000); // refreshes with a interval of 1 second
   return cmds_interval[id];
}

function exec(cmd, callback){
  $.ajax({
    url:"daemon.php",
    dataType: "text",
    data: {"daemon":cmd},
    success: function(id){
       receiveResult(cmd, id, callback);
    }
  });
}

Example of use:

In HTML:

<pre id="console"></pre>
<script language="javascript" type="text/javascript" src="path/to/jquery.js"></script>
<script language="javascript" type="text/javascript" src="path/to/daemon.js"></script>
<script language="javascript" type="text/javascript" src="path/to/demo.js"></script>

In demo.js: (Javascript, and uses jQuery too):

exec("python script.py", function(msg){ 
    $("#console").html(msg);
});

This should work. If it not works, await to tomorrow as i'm exiting now. Good luck.

PS: If the code not work, you can see the code as a example of a algorithm to what you want.

PS 2: The execInBackground function is from here: http://www.php.net/manual/en/function.exec.php#86329

Thank you Fernando! I am reading [the fine manual](http://php.net/manual/en/function.proc-open.php) for `proc_open` but even with the examples shown I cannot seem to figure out where the code should 'get' the additional output from. The AJAX call obviously cannot call the running instance of the script. There exists no 'push' functionality from the running PHP script to AJAX that I know of. I accept your generous offer of codes, in fact I'll put a bounty on this as soon as SO lets me as I usually never ask for codes but I'm genuinely stuck on this one. Thanks. — dotancohen, May 23 '13 at 15:06
@dotancohen The recommended in this case is use some sort of external script, i edit the answer to have some sort of code that can works in your case with Ajax and some sort of log if possible. — Fernando Jorge Mota, May 23 '13 at 22:47
@dotancohen This helps you? Please make me updated in relation to this question/answer. ;) — Fernando Jorge Mota, May 24 '13 at 23:33
Hi Fernando! I'll be in a position to test this next week, and I'll get back to it. I'm sorry, but something has come up and I've been forced to take off from this. But I will get right back to it as soon as possible. — dotancohen, Jun 05 '13 at 19:11

score 1 · Answer 2 · answered May 23 '13 at 15:05

You could use proc_open to do this.

But if you want to go with the file alternative, there's no need to modify your python script, you can simply output to a file.

$execution = shell_exec("script.py > /ouput/file/path &");

and then read the file through ajax.

Provide feedback from Python script to PHP

2 Answers2