How to run the Python interpreter and get its output using Java?

Question

Is it possible to get console output from Python using Java? Here is an example of such output:

Python 3.3.4 (v3.3.4:7ff62415e426, Feb 10 2014, 18:13:51) [MSC v.1600 64 bit (AMD64)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> 2+2
4
>>>

Now, the main goal is to get the above output by calling Python interpreter using Java. Here's my attempt:

//...
//Irrelevant code omitted

ProcessBuilder processBuilder = new ProcessBuilder("cmd");
processBuilder.redirectErrorStream(true);
processBuilder.start();
processBuilder.command("python2");
Process pythonProcess = processBuilder.start();
OutputStream outputStream = pythonProcess.getOutputStream();
OutputStreamWriter osw = new OutputStreamWriter(outputStream);
osw.write("2+2\r\nquit()\r\n");
osw.flush();
osw.close();
InputStream inputStream = pythonProcess.getInputStream();
BufferedReader bufferedReader = new BufferedReader(
                  new InputStreamReader(inputStream));
String line;

while( (line=bufferedReader.readLine())!=null) {

    System.out.println(line);

}

//...
//Irrelevant code omitted

I understand that calling the start method spawns a new process with its environment for execution. Writing python2 to a process's output stream results in creating another process. This is when the problem starts. I haven't been able to find a way of sending the command 2+2 to the Python interpreter (which is the child process of CMD) instead of its parent process.

To summarize: How do I run the Python interpreter, execute some commands inside it, and in the end, print the results to the standard output?

Answer 1

The Python executable can tell that you are running the command non-interactively. Once it realises it's being run non-interactively it will no longer attempt to interact with you; why bother printing to stdout or reading from stdin if noone is there?

To see that this is true you would attempt to run e.g. "ls" or "ps" and see that they work in your program, but then run e.g. "ftp" or "telnet" or "python" and see that don't work and output nothing.

In the parlance of Linux the problem is that the way we're running processes does not attach a TTY to them. The solution is to trick them into believing there's a TTY on the other end by creating a PTY.

Trick an application into thinking its stdin is interactive, not a pipe

On:

• my Mac OS X 10.9.4 laptop with CPython 2.7.5 and Java 1.8.0_05 and
• a Ubuntu 12.04.4 LTS server with CPython 2.7.5 and Java 1.7.0_55

the following works, albeit in a very ugly fashion:

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.IOException;
import java.io.OutputStream;
import java.io.OutputStreamWriter;

class Foo {
    public static void main(String[] args) throws IOException, InterruptedException {
        // https://stackoverflow.com/questions/1401002/trick-an-application-into-thinking-its-stdin-is-interactive-not-a-pipe
        // 
        // Using script or unbuffer is the important catch. Without this
        // step you cannot use stdin in Python interactively, even with
        // python -u. At least script comes with Linux/Mac OS X, but
        // unbuffer works fine too.
        ProcessBuilder pb;
        switch(System.getProperty("os.name")) {
            case "Mac OS X":
                pb = new ProcessBuilder(
                    "/usr/bin/script", "-q", "/dev/null", "/usr/bin/python");
                break;
            default:
                // Linux
                pb = new ProcessBuilder(
                    "/usr/bin/script", "-qfc", "/usr/bin/python", "/dev/null");

        }
        // This doesn't make a difference.
        // pb.redirectErrorStream(true);

        Process p = pb.start();

        char[] readBuffer = new char[1000];
        InputStreamReader isr = new InputStreamReader(p.getInputStream());
        BufferedReader br = new BufferedReader(isr);
        int charCount;
        boolean written = false;
        while(true) {
            if (!br.ready() && !written) {
                // Ugly. Should be reading for '>>>' prompt then writing.
                Thread.sleep(1000);
                if (!written) {
                    written = true;
                    OutputStream os = p.getOutputStream();
                    OutputStreamWriter osw = new OutputStreamWriter(os);
                    BufferedWriter bw = new BufferedWriter(osw);
                    bw.write("2+2");
                    bw.newLine();
                    bw.write("quit()");
                    bw.newLine();
                    bw.flush();
                    bw.close();
                }
                continue;
            }
            charCount = br.read(readBuffer);
            if (charCount > 0)
                System.out.print(new String(readBuffer, 0, charCount));
            else
                break;
        }
    }
}

I wouldn't do it like this. Instead I'd use threads to stay interactive and avoid blocking on reads and do what expect does, which is wait for certain prompts before writing out. In the code above I blindly sleep then hope for the best.

However I notice that you're using Windows, since you've run "cmd". I don't know how to create PTYs on Windows, sorry. I think though that you can get Expect for Windows; unbuffer is a utility within Expect:

http://expect.sourceforge.net/

Or try cygwin, but again I haven't tested this.

For further background on TTYs and PTYs see:

How to create a pseudo-tty for reading output and writing to input

Answer 2

The answer from Asim Ihsan has the right root cause - python will know that it's not connected to an interactive terminal and not behave the way that you want. You can make python act in interactive mode, however, by passing the -i flag when you start python. This is simpler than what Asim proposes. You should just start python directly (no need the intermediary of starting cmd first). You'll use something like

ProcessBuilder pb = new ProcessBuilder('python', '-i');

and then proceed more or less as in the original question. Start the process, get the associated streams, and read and write to the streams.