59

I'm just experimenting with PHP and shell_exec on my Linux server. It's a really cool function to use and I am really enjoying it so far. Is there a way to view the live output that is going on while the command is running?

For example, if ping stackoverflow.com was run, while it is pinging the target address, every time it pings, show the results with PHP? Is that possible?

I would love to see the live update of the buffer as it's running. Maybe it's not possible but it sure would be nice.

This is the code I am trying and every way I have tried it always displays the results after the command is finished.

<?php

  $cmd = 'ping -c 10 127.0.0.1';

  $output = shell_exec($cmd);

  echo "<pre>$output</pre>";

?>

I've tried putting the echo part in a loop but still no luck. Anyone have any suggestions on making it show the live output to the screen instead of waiting until the command is complete?

I've tried exec, shell_exec, system, and passthru. Everyone of them displays the content after it's finished. Unless I'm using the wrong syntax or I'm not setting up the loop correctly.

Henders
  • 1,195
  • 1
  • 21
  • 27
Palumbo Software
  • 593
  • 1
  • 5
  • 5

3 Answers3

121

To read the output of a process, popen() is the way to go. Your script will run in parallel with the program and you can interact with it by reading and writing it's output/input as if it was a file.

But if you just want to dump it's result straight to the user you can cut to the chase and use passthru():

echo '<pre>';
passthru($cmd);
echo '</pre>';

If you want to display the output at run time as the program goes, you can do this:

while (@ ob_end_flush()); // end all output buffers if any

$proc = popen($cmd, 'r');
echo '<pre>';
while (!feof($proc))
{
    echo fread($proc, 4096);
    @ flush();
}
echo '</pre>';

This code should run the command and push the output straight to the end user at run time.

More useful information

Note that if you are using sessions then having one of those running will prevent the user from loading other pages, as sessions enforce that concurrent requests cannot happen. To prevent this from being a problem, call session_write_close() before the loop.

If your server is behind a nginx gateway, then the nginx buffering may be disruptive to the desired behavior. Set the header header('X-Accel-Buffering: no'); to hint nginx that it shouldn't do that. As headers are sent first, this has to be called in the beginning of the script, before any data is sent.

Havenard
  • 27,022
  • 5
  • 36
  • 62
  • I want to be able to see everything it's doing just as if I was in terminal and typed the command. Everything the terminal window sees is what I want to display to my page. – Palumbo Software Nov 21 '13 at 00:11
  • 1
    I actually found a code when I was researching that was similar to that but i must have messed it up. I tried for about an hour to get it to work. Thank you that's exactly what I wanted that works perfect. – Palumbo Software Nov 21 '13 at 00:22
  • 2
    Yeah, the trick is to enforce that no buffering will happen at all. This code makes sure of that. Usually PHP will only flush output when it accumulates at least 4KB of data. – Havenard Nov 21 '13 at 00:29
  • @Havenard Will this echo live output on a web page? I tried this code but it is not happening! thank u this is the question : http://stackoverflow.com/questions/20248493/php-outputting-the-system-shell-exec-command-output-in-web-browser – user2723949 Nov 28 '13 at 12:14
  • This worked.. for a bit, but then the wget process I was running with it stops after a while.. not sure why as it's still running but just not responding.. could this method be a memory hog? – John Hunt Sep 16 '15 at 14:22
  • @JohnHunt As you can see the basic premise for this to work is to not have buffering, so no, on the contrary, this is avoiding to use memory. – Havenard Sep 16 '15 at 16:43
  • @Havenard Yup, it was just wget getting stuck on a bad DNS resolution - unfortunately the old wget doesn't tell you what's going on even in verbose mode. – John Hunt Sep 17 '15 at 09:45
  • 2
    When capturing output from a python process, you'll need to disable output buffering: python -u /path/to/watch ... or PYTHONUNBUFFERED=1 watch ... – Umair Ayub Jan 21 '18 at 10:05
  • alright, to get this clear, this only works for commands which finishes eventually right? If I see the `feof` check... Cause personally I was looking for something which could live display the output of a `ping` command for example. I have commands which don't end until you kill the process, that's why. Any solution for this as well ? :) – Erik van de Ven Jul 16 '18 at 19:51
  • @ErikvandeVen Kill the process. v0v – Havenard Jul 16 '18 at 21:23
  • @ErikvandeVen You could like, periodically check for a flag stored in `$_SESSION` to determine if your loop should be terminated, this way the user can send an order to interrupt the process via another request. But for this to work you must make sure the session is closed with `session_write_close()` because sessions don't allow concurrency, the user won't be able to load any page that calls `session_start()` while a script with that session is still running. – Havenard Jul 16 '18 at 21:27
  • Thanks for your response Havenard! Sounds good, will look into that! Guess I found a solution on how to read the live data while the process is still running, as well. Instead of restarting the process using `shell_exec` every single time, I could write the output to a text file I suppose and keep calling that file periodically, or use tail in a way like this https://stackoverflow.com/a/36647934/1843511. – Erik van de Ven Jul 17 '18 at 07:10
  • 1
    @ErikvandeVen In that case you might want to take a look into `mkfifo()`. – Havenard Jul 17 '18 at 14:43
  • 4
    If you're in Nginx, you should also add `header('X-Accel-Buffering: no');` to make sure gzip and fastcgi_buffering are off for that individual response. I was wondering why this code didn't work, and that was why. – RaisinBranCrunch Jul 18 '19 at 19:52
  • I'm wondering how to handle this if 4096 is not enough length – zyrup Aug 08 '22 at 14:11
  • @zyrup It's always enough, because it's a loop, so it will read 4KB at a time until it's read it all. – Havenard Aug 08 '22 at 19:52
  • @Havenard but what if a line is longer than 4096 bytes? – zyrup Aug 15 '22 at 13:28
  • 1
    @zyrup It doesn't matter. You could very well read 1 byte at a time and it would still work, just inefficiently. – Havenard Aug 15 '22 at 19:05
  • @Havenard so I recently was running a command that usually would return 300k+ characters but could not process the output properly because the command was limited by 4096. 8192 seems to be the limit https://stackoverflow.com/questions/14733400/php-fread-limited-to-8192-bytes – zyrup Aug 28 '22 at 14:52
  • 1
    @zyrup Whenever you're dealing with data streams, from files or otherwise, you will have to account for the case where the data you received is incomplete, buffer it and read the rest before doing something with it. This is standard when dealing with data from network for instance, each one of a sender's `send()` does not account for receiver's `recv()`, the data may arrive either concatenated or fragmented. In this case you will have to deal with something similar, since the program is generating the output in real time. – Havenard Aug 29 '22 at 00:15
  • 1
    @zyrup Usually what they do is to either send a header containing the length of packet beforehand, or put a marker in the end to signal the packet is ended. HTTP for example uses both. Because the HTTP headers have variable length, it sends an empty line after the headers to signal the headers were completely sent, and it includes in the headers the length of the data that follows. When the length isn't available it signals the end by terminating the connection. – Havenard Aug 29 '22 at 00:17
26

First of all, thanks Havenard for your snippet - it helped a lot!

A slightly modified version of Havenard's code which i found useful.

<?php
/**
 * Execute the given command by displaying console output live to the user.
 *  @param  string  cmd          :  command to be executed
 *  @return array   exit_status  :  exit status of the executed command
 *                  output       :  console output of the executed command
 */
function liveExecuteCommand($cmd)
{

    while (@ ob_end_flush()); // end all output buffers if any

    $proc = popen("$cmd 2>&1 ; echo Exit status : $?", 'r');

    $live_output     = "";
    $complete_output = "";

    while (!feof($proc))
    {
        $live_output     = fread($proc, 4096);
        $complete_output = $complete_output . $live_output;
        echo "$live_output";
        @ flush();
    }

    pclose($proc);

    // get exit status
    preg_match('/[0-9]+$/', $complete_output, $matches);

    // return exit status and intended output
    return array (
                    'exit_status'  => intval($matches[0]),
                    'output'       => str_replace("Exit status : " . $matches[0], '', $complete_output)
                 );
}
?>

Sample Usage :

$result = liveExecuteCommand('ls -la');

if($result['exit_status'] === 0){
   // do something if command execution succeeds
} else {
    // do something on failure
}
Amith
  • 6,818
  • 6
  • 34
  • 45
  • Would something like this work for an entire .sh script? I want to get the immediate feedback of a .sh script called from PHP? See my question here: http://stackoverflow.com/questions/34999905/keep-bash-shell-functions-between-shell-exec-requests – DevelumPHP Jan 25 '16 at 22:01
  • It would work. Something like : liveExecuteCommand("sh /path/to/script.sh"); – Amith Jan 28 '16 at 15:24
  • 1
    fantastic ... exactly what I was looking for. – frabjous Sep 11 '16 at 13:48
  • one small suggestion though: throw an intval( ... ) on the exit_status to return it as an integer rather than a string. – frabjous Sep 11 '16 at 14:01
  • 1
    Very usefull. Please not that you should *not* write your shell command with a trailing semicolon, otherwise the `exit_status` will always be 0. – 4wk_ May 23 '17 at 15:46
  • When capturing output from a python process, you'll need to disable output buffering: python -u /path/to/watch ... or PYTHONUNBUFFERED=1 watch ... – Umair Ayub Jan 21 '18 at 10:05
  • Umair, your comment saved my day. Also, i modified `$live_output= fread($proc, 256);` to `$live_output=htmlspecialchars(fread($proc, 256));` to avoid troubles when a `<` appears. – Rainer Glüge Aug 08 '18 at 09:15
  • `htmlspecialchars` may be too much, `str_replace("<", "&lt", fread($proc, 256));` may be sufficient – Rainer Glüge Aug 08 '18 at 13:43
7

If you're willing to download a dependency, Symfony's processor component does this. I found the interface to working with this cleaner than reinventing anything myself with popen() or passthru().

This was provided by the Symfony documentation:

You can also use the Process class with the foreach construct to get the output while it is generated. By default, the loop waits for new output before going to the next iteration:

$process = new Process('ls -lsa');
$process->start();

foreach ($process as $type => $data) {
    if ($process::OUT === $type) {
        echo "\nRead from stdout: ".$data;
    } else { // $process::ERR === $type
        echo "\nRead from stderr: ".$data;
    }
}

As a warning, I've run into some problems PHP and Nginx trying to buffer the output before sending it to the browser. You can disable output buffering in PHP by turning it off in php.ini: output_buffering = off. There's apparently a way to disable it in Nginx, but I ended up using the PHP built in server for my testing to avoid the hassle.

I put up a full example of this on Gitlab: https://gitlab.com/hpierce1102/web-shell-output-streaming

HPierce
  • 7,249
  • 7
  • 33
  • 49
  • 1
    Yes, in Nginx you should also do `header('X-Accel-Buffering: no');` to make sure gzip and fastcgi_buffering are off for that individual response, instead of turning it off for all requests. Took me too long to figure that out. – RaisinBranCrunch Jul 18 '19 at 19:50