Making a system call that returns the stdout output as a string

Question

Perl and PHP do this with backticks. For example,

$output = `ls`;

Returns a directory listing. A similar function, system("foo"), returns the operating system return code for the given command foo. I'm talking about a variant that returns whatever foo prints to stdout.

How do other languages do this? Is there a canonical name for this function? (I'm going with "backtick"; though maybe I could coin "syslurp".)

Answer 1

Python

from subprocess import check_output as qx

output = qx(['ls', '-lt'])

Python <2.7 or <3.1

Extract subprocess.check_output() from subprocess.py or adapt something similar to:

import subprocess

def cmd_output(args, **kwds):
  kwds.setdefault("stdout", subprocess.PIPE)
  kwds.setdefault("stderr", subprocess.STDOUT)
  p = subprocess.Popen(args, **kwds)
  return p.communicate()[0]

print cmd_output("ls -lt".split())

The subprocess module has been in the stdlib since 2.4.

Answer 2

Python:

import os
output = os.popen("foo").read()

Answer 3

[At the request of Alexman and dreeves -- see comments --, you will find at this DZones Java Snippet page a full version Os-independent for making, in this instance, a 'ls'. This is a direct answer to their code-challenge.
What follows below is just the core: Runtime.exec, plus 2 thread to listen to stdout and stderr. ]

Java "Simple!":

E:\classes\com\javaworld\jpitfalls\article2>java GoodWindowsExec "dir *.java"
Executing cmd.exe /C dir *.java
...

Or in java code

String output = GoodWindowsExec.execute("dir");

But to do that, you need to code...
... this is embarrassing.

import java.util.*;
import java.io.*;
class StreamGobbler extends Thread
{
    InputStream is;
    String type;
    StringBuffer output = new StringBuffer();

    StreamGobbler(InputStream is, String type)
    {
        this.is = is;
        this.type = type;
    }

    public void run()
    {
        try
        {
            InputStreamReader isr = new InputStreamReader(is);
            BufferedReader br = new BufferedReader(isr);
            String line=null;
            while ( (line = br.readLine()) != null)
                System.out.println(type + ">" + line);
                output.append(line+"\r\n")
            } catch (IOException ioe)
              {
                ioe.printStackTrace();  
              }
    }
    public String getOutput()
    {
        return this.output.toString();
    }
}
public class GoodWindowsExec
{
    public static void main(String args[])
    {
        if (args.length < 1)
        {
            System.out.println("USAGE: java GoodWindowsExec <cmd>");
            System.exit(1);
        }
    }
    public static String execute(String aCommand)
    {
        String output = "";
        try
        {            
            String osName = System.getProperty("os.name" );
            String[] cmd = new String[3];
            if( osName.equals( "Windows 95" ) )
            {
                cmd[0] = "command.com" ;
                cmd[1] = "/C" ;
                cmd[2] = aCommand;
            }
            else if( osName.startsWith( "Windows" ) )
            {
                cmd[0] = "cmd.exe" ;
                cmd[1] = "/C" ;
                cmd[2] = aCommand;
            }

            Runtime rt = Runtime.getRuntime();
            System.out.println("Executing " + cmd[0] + " " + cmd[1] 
                               + " " + cmd[2]);
            Process proc = rt.exec(cmd);
            // any error message?
            StreamGobbler errorGobbler = new 
                StreamGobbler(proc.getErrorStream(), "ERROR");            

            // any output?
            StreamGobbler outputGobbler = new 
                StreamGobbler(proc.getInputStream(), "OUTPUT");

            // kick them off
            errorGobbler.start();
            outputGobbler.start();

            // any error???
            int exitVal = proc.waitFor();
            System.out.println("ExitValue: " + exitVal);   

            output = outputGobbler.getOutput();
            System.out.println("Final output: " + output);   

        } catch (Throwable t)
          {
            t.printStackTrace();
          }
        return output;
    }
}

Answer 4

Yet another way to do it in Perl (TIMTOWTDI)

$output = <<`END`;
ls
END

This is specially useful when embedding a relatively large shell script in a Perl program

Answer 5

Ruby: either backticks or the '%x' builtin syntax.

puts `ls`;
puts %x{ls};

Answer 6

An alternative method in perl

$output = qx/ls/;

This had the advantage that you can choose your delimiters, making it possible to use ` in the command (though IMHO you should reconsider your design if you really need to do that). Another important advantage is that if you use single quotes as delimiter, variables will not be interpolated (a very useful)

Answer 7

Haskell:

import Control.Exception
import System.IO
import System.Process
main = bracket (runInteractiveCommand "ls") close $ \(_, hOut, _, _) -> do
    output <- hGetContents hOut
    putStr output
  where close (hIn, hOut, hErr, pid) =
          mapM_ hClose [hIn, hOut, hErr] >> waitForProcess pid

With MissingH installed:

import System.Cmd.Utils
main = do
    (pid, output) <- pipeFrom "ls" []
    putStr output
    forceSuccess pid

This is an easy operation in "glue" languages like Perl and Ruby, but Haskell isn't.

Answer 8

In shell

OUTPUT=`ls`

or alternatively

OUTPUT=$(ls)

This second method is better because it allows nesting, but isn't supported by all shells, unlike the first method.

Answer 9

Erlang:

os:cmd("ls")

Answer 10

Well, since this is system dependent, there are many languages that do not have a built-in wrapper for the various system calls needed.

For example, Common Lisp itself was not designed to run on any specific system. SBCL (the Steel Banks Common Lisp implementation), though, does provide an extension for Unix-like systems, as do most other CL implementations. This is much more "mighty" than just getting the output, of course (you have control over the running process, can specify all kinds of stream directions, etc., confer to the SBCL manual, chapter 6.3), but it is easy to write a little macro for this specific purpose:

(defmacro with-input-from-command ((stream-name command args) &body body)
  "Binds the output stream of command to stream-name, then executes the body
   in an implicit progn."
  `(with-open-stream
       (,stream-name
         (sb-ext:process-output (sb-ext:run-program ,command
                                                    ,args
                                                    :search t
                                                    :output :stream)))
     ,@body))

Now, you can use it like this:

(with-input-from-command (ls "ls" '("-l"))
  ;;do fancy stuff with the ls stream
  )

Perhaps you want to slurp it all into one string. The macro is trivial (though perhaps more concise code is possible):

(defmacro syslurp (command args)
  "Returns the output from command as a string. command is to be supplied
   as string, args as a list of strings."
  (let ((istream (gensym))
        (ostream (gensym))
        (line (gensym)))
    `(with-input-from-command (,istream ,command ,args)
       (with-output-to-string (,ostream)
         (loop (let ((,line (read-line ,istream nil)))
                 (when (null ,line) (return))
                 (write-line ,line ,ostream)))))))

Now you can get a string with this call:

(syslurp "ls" '("-l"))

Answer 11

Mathematica:

output = Import["!foo", "Text"];

Answer 12

Yet another way (or 2!) in Perl....

open my $pipe, 'ps |';
my @output = < $pipe >;
say @output;

open can also be written like so...

open my $pipe, '-|', 'ps'

Answer 13

Years ago I wrote a plugin for jEdit that interfaced to a native application. This is what I used to get the streams off the running executable. Only thing left to do is while((String s = stdout.readLine())!=null){...}:

/* File:    IOControl.java
 *
 * created: 10 July 2003
 * author:  dsm
 */
package org.jpop.io;

import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.PrintStream;

/**
 *  Controls the I/O for a process. When using the std[in|out|err] streams, they must all be put on
 *  different threads to avoid blocking!
 *
 * @author     dsm
 * @version    1.5
 */
public class IOControl extends Object {
    private Process process;
    private BufferedReader stdout;
    private BufferedReader stderr;
    private PrintStream stdin;

    /**
     *  Constructor for the IOControl object
     *
     * @param  process  The process to control I/O for
     */
    public IOControl(Process process) {
        this.process = process;
        this.stdin = new PrintStream(process.getOutputStream());
        this.stdout = new BufferedReader(new InputStreamReader(process.getInputStream()));
        this.stderr = new BufferedReader(new InputStreamReader(process.getErrorStream()));
    }

    /**
     *  Gets the stdin attribute of the IOControl object
     *
     * @return    The stdin value
     */
    public PrintStream getStdin() {
        return this.stdin;
    }

    /**
     *  Gets the stdout attribute of the IOControl object
     *
     * @return    The stdout value
     */
    public BufferedReader getStdout() {
        return this.stdout;
    }

    /**
     *  Gets the stderr attribute of the IOControl object
     *
     * @return    The stderr value
     */
    public BufferedReader getStderr() {
        return this.stderr;
    }

    /**
     *  Gets the process attribute of the IOControl object. To monitor the process (as opposed to
     *  just letting it run by itself) its necessary to create a thread like this: <pre>
     *. IOControl ioc;
     *.
     *. new Thread(){
     *.     public void run(){
     *.         while(true){    // only necessary if you want the process to respawn
     *.             try{
     *.                 ioc = new IOControl(Runtime.getRuntime().exec("procname"));
     *.                 // add some code to handle the IO streams
     *.                 ioc.getProcess().waitFor();
     *.             }catch(InterruptedException ie){
     *.                 // deal with exception
     *.             }catch(IOException ioe){
     *.                 // deal with exception
     *.             }
     *.
     *.             // a break condition can be included here to terminate the loop
     *.         }               // only necessary if you want the process to respawn
     *.     }
     *. }.start();
     *  </pre>
     *
     * @return    The process value
     */
    public Process getProcess() {
        return this.process;
    }
}

Answer 14

Don't forget Tcl:

set result [exec ls]

Answer 15

C# 3.0, less verbose than this one:

using System;
using System.Diagnostics;

class Program
{
    static void Main()
    {
        var info = new ProcessStartInfo("cmd", "/c dir") { UseShellExecute = false, RedirectStandardOutput = true };
        Console.WriteLine(Process.Start(info).StandardOutput.ReadToEnd());
    }
}

Caveat: Production code should properly dispose the Process object...

Answer 16

Here's another Lisp way:

(defun execute (program parameters &optional (buffer-size 1000))
  (let ((proc (sb-ext:run-program program parameters :search t :output :stream))
        (output (make-array buffer-size :adjustable t :fill-pointer t 
                            :element-type 'character)))
    (with-open-stream (stream (sb-ext:process-output proc))
      (setf (fill-pointer output) (read-sequence output stream)))
    output))

Then, to get your string:

(execute "cat" '("/etc/hosts"))

If you want to run a command that creates prints a great deal of info to STDOUT, you can run it like this:

(execute "big-writer" '("some" "parameters") 1000000)

The last parameter preallocates a large amount of space for the output from big-writer. I'm guessing this function could be faster than reading the output stream one line at a time.

Answer 17

Lua:

    foo = io.popen("ls"):read("*a")

Answer 18

Perl:

$output = `foo`;

ADDED: This is really a multi-way tie. The above is also valid PHP, and Ruby, for example, uses the same backtick notation as well.

Answer 19

In PHP

$output = `ls`;

or

$output = shell_exec('ls');

Answer 20

C (with glibc extension):

#define _GNU_SOURCE
#include <stdio.h>
int main() {
    char *s = NULL;
    FILE *p = popen("ls", "r");
    getdelim(&s, NULL, '\0', p);
    pclose(p);
    printf("%s", s);
    return 0;
}

Okay, not really concise or clean. That's life in C...

Answer 21

In C on Posix conformant systems:

#include <stdio.h> 

FILE* stream = popen("/path/to/program", "rw");
fprintf(stream, "foo\n"); /* Use like you would a file stream. */
fclose(stream);

Answer 22

Why there is still no c# guy here :)

This is how to do it in C#. The built-in way.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;

namespace TestConsole
{
    class Program
    {
        static void Main(string[] args)
        {
            Process p = new Process();

            p.StartInfo.UseShellExecute = false;
            p.StartInfo.CreateNoWindow = true;
            p.StartInfo.RedirectStandardOutput = true;
            p.StartInfo.RedirectStandardError = true;
            p.StartInfo.FileName = "cmd";
            p.StartInfo.Arguments = "/c dir";
            p.Start();

            string res = p.StandardOutput.ReadToEnd();
            Console.WriteLine(res);
        }

    }
}

Answer 23

Clozure Common Lisp:

(with-output-to-string (stream)
   (run-program "ls" '("-l") :output stream))

LispWorks

(with-output-to-string (*standard-output*)
  (sys:call-system-showing-output "ls -l" :prefix "" :show-cmd nil))

Answer 24

J:

output=:2!:0'ls'

Answer 25

Perl, another way:

use IPC::Run3

my ($stdout, $stderr);
run3 ['ls'], undef, \$stdout, \$stderr
    or die "ls failed";

Useful because you can feed the command input, and get back both stderr and stdout separately. Nowhere near as neat/scary/slow/disturbing as IPC::Run, which can set up pipes to subroutines.

Answer 26

Icon/Unicon:

stream := open("ls", "p")
while line := read(stream) do { 
    # stuff
}

The docs call this a pipe. One of the good things is that it makes the output look like you're just reading a file. It also means you can write to the app's stdin, if you must.

Answer 27

Granted, it is not the smaller ( from all the languages available ) but it shouldn't be that verbose.

This version is dirty. Exceptions should be handled, reading may be improved. This is just to show how a java version could start.

Process p = Runtime.getRuntime().exec( "cmd /c " + command );
InputStream i = p.getInputStream();
StringBuilder sb = new StringBuilder();
for(  int c = 0 ; ( c =  i.read() ) > -1  ; ) {
    sb.append( ( char ) c );
}

Complete program below.

import java.io.*;

public class Test { 
    public static void main ( String [] args ) throws IOException { 
        String result = execute( args[0] );
        System.out.println( result );
    }
    private static String execute( String command ) throws IOException  { 
        Process p = Runtime.getRuntime().exec( "cmd /c " + command );
        InputStream i = p.getInputStream();
        StringBuilder sb = new StringBuilder();
        for(  int c = 0 ; ( c =  i.read() ) > -1  ; ) {
            sb.append( ( char ) c );
        }
        i.close();
        return sb.toString();
    }
}

Sample ouput ( using the type command )

C:\oreyes\samples\java\readinput>java Test "type hello.txt"
This is a sample file
with some
lines

Sample output ( dir )

 C:\oreyes\samples\java\readinput>java Test "dir"
 El volumen de la unidad C no tiene etiqueta.
 El número de serie del volumen es:

 Directorio de C:\oreyes\samples\java\readinput

12/16/2008  05:51 PM    <DIR>          .
12/16/2008  05:51 PM    <DIR>          ..
12/16/2008  05:50 PM                42 hello.txt
12/16/2008  05:38 PM             1,209 Test.class
12/16/2008  05:47 PM               682 Test.java
               3 archivos          1,933 bytes
               2 dirs            840 bytes libres

Try any

java Test netstat
java Test tasklist
java Test "taskkill /pid 416"

EDIT

I must admit I'm not 100% sure this is the "best" way to do it. Feel free to post references and/or code to show how can it be improved or what's wrong with this.