How to run a subprocess in the background python

Question

I'm writing an application in python that initiates a JVM in java by calling a shell script using a python subprocess. However, my problem is that the correct way I have it written, the JVM starts and blocks the rest of the processes that happen after it. I need the JVM to run while I'm calling another function, and I need to stop the JVM after the process has finished running.

Python code:

process = subprocess.Popen('runJVM.sh', shell=True, stderr=subprocess.STDOUT)
process.wait()

r = Recommender()
r.load()
assert len(sys.argv) > 1, '%d arguments supplied, one needed' %(len(sys.argv)-1)
print "recommendations" + str(r.get_users_recommendation(sys.argv[1:]))

....
def get_users_recommendation(self, user_list):
    j_id_list = ListConverter().convert(class_list, self.gateway._gateway_client)
    recs = self.gateway.entry_point.recommend(j_id_list)
    return recs

Where:

 from py4j.java_gateway import JavaGateway
 self.gateway = JavaGateway()

I can't get the get_user_recommendations to run because the JVM server is blocking the process. How do I not make it block the rest of the Python script and then kill it once the python methods have finished running and returned a value? Much thanks.

Answer 1

process.wait() is blocking your process. Remove that line and the rest of the code will run. Then you can make it end by calling Popen.terminate()

Popen is invoking CreateProcess, that means the OS launches a new process and returns the PID to be stored on your process variable.

If you use process.wait(), then your code will wait until the recently created process ends (it never ends in this case unless you terminate it externally).

If you do not call process.wait(), your code will continue to execute. But you still have control over the other process and you can do things like process.terminate() or even process.kill() if necessary.