JAVA interaction with CPython

Question

My Objective:

When I run Python (CPython) from command line (not Jython since it does not support some packages like NumPy) I can interactively write lines of code and see results from its output.

My objective is to do this programmatically in JAVA. Here is my attempt to do so:

Code:

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;

public class PythonProcess {
    private Process proc;
    private BufferedReader stdInput;
    private BufferedReader stdError;
    private BufferedWriter stdOutput;

    public PythonProcess() {
        Runtime rt = Runtime.getRuntime();
        String[] commands = { "python.exe" };
        try {
            proc = rt.exec(commands);
            stdInput = new BufferedReader(new InputStreamReader(proc.getInputStream()));
            stdError = new BufferedReader(new InputStreamReader(proc.getErrorStream()));
            stdOutput = new BufferedWriter(new OutputStreamWriter(proc.getOutputStream()));
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    private String executeCommand(String command) throws IOException {
        stdOutput.write(command + "\n");
        stdOutput.newLine();
        String s = null;
        StringBuffer str = new StringBuffer();
        while ((s = stdInput.readLine()) != null) {
            str.append(s);
        }
        return str.toString();
    }

    public void initialize() throws IOException {
        // will create a file - if correctly executed in python 
        stdOutput.write("f = open('c:/downloads/deleteme.txt','w')");
        stdOutput.newLine();
        stdOutput.write("f.write('hi there, I am Python \n')");
        stdOutput.newLine();
        stdOutput.write("f.close()");
        stdOutput.newLine();
        stdOutput.flush();
    }

    public static void main(String[] args) throws IOException {
        PythonProcess proc = new PythonProcess();
        proc.initialize(); // time demanding initialization
        for (int i = 0; i < 10; i++) {
            String out = proc.executeCommand("print \"Hello from command line #"+i+"\"");
            System.out.println("Output: " + out);
        }
    }
}

Problem:

It seems that the code passed by stdOutput in initialize() method is not executed by Python at all since the file c:/downloads/deleteme.txt was not created. Later on I am also unable to read any output from the stdInput when calling executeCommand method.

Questions:

1. Is there any simple way how to fix the code?
2. Can anyone point me to some example how can I interact with python e.g. by client - server way
3. or Any other idea?

---

BTW1: Jython is not the way to go since I need to execute CPython directives that are not supported by python.

---

BTW2: I know that I can execute the python script in noninteractive way by String[] commands = { "python.exe" "script.py"};. The issue is when the initilaization takes significant time it would mean significant performance issue.

Answer 1

Direct interaction with Python process is unfortunately not possible. Instead of it is a solution using TCP sockets:

PythonServer.java

package pythonexample;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;

public class PythonServer {
    public static void main(String[] args) throws IOException, InterruptedException {
        ProcessBuilder   ps=new ProcessBuilder("python.exe","tcpServer.py");
        ps.redirectErrorStream(true);
        System.out.println("Starting Python server.");
        Process pr = ps.start();  

        BufferedReader in = new BufferedReader(new InputStreamReader(pr.getInputStream()));
        String line;
        while ((line = in.readLine()) != null) {
            System.out.println(line);
        }
        pr.waitFor();

        in.close();
        System.exit(0);
    }
}

tcpServer.py

import socket

# DO INITIALIZATION

TCP_IP = '127.0.0.1'
TCP_PORT = 5006
BUFFER_SIZE = 20  # Normally 1024, shorter if we want fast response

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)

while 1:
    print 'Waiting for connection'
    conn, addr = s.accept()
    print 'Connection address:', addr
    if addr[0] == '127.0.0.1': # security - only local processes can connect
        data = conn.recv(BUFFER_SIZE)
        print "received data:", data
        res = eval(data)
        print "Sending result:", res
        conn.send("This is reply from Python " + str(res) + "\n")
conn.close()

PythonClient.java

package pythonexample;

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.net.Socket;

public class PythonClient {
    public static void main(String[] args) throws IOException {
        String serverAddress = "127.0.0.1";
        Socket s = new Socket(serverAddress, 5006);

        // do this in loop if needed
        String result = sendCommand(s, "1+1                                    \n");
        System.out.println("Received result: "+result);

        // closing socket 
        s.close();
    }

    private static String sendCommand(Socket s, String command) throws IOException {
        BufferedWriter ouput = new BufferedWriter(new OutputStreamWriter(s.getOutputStream()));
        ouput.write(command);
        ouput.flush();

        BufferedReader input = new BufferedReader(new InputStreamReader(s.getInputStream()));
        String answer = input.readLine();
        return answer;
    }
}

First of all execute PythonServer class, which initializes the server. Than execute PythonClient which sends data to servers and gets result.

Of course all the features like NumPy and other which are not supported by e.g. Jython are now available.