ProcessBuilder : inheritIO not working when python with Thread setDaemon

Question

I am running python process from Java (openjdk 1.7.0_85):

    ProcessBuilder pb = new ProcessBuilder(bin, processJar, Integer.toString(port), Integer.toString(portRev), path, extl, thriftc, thrifts, thriftd);
    pb.inheritIO();
    pb.redirectErrorStream(true);
    pb.directory(new File(dir));
    try {
        logger.trace("Start process");
        lock.lock();
        process = pb.start();
        cond.await();
        lock.unlock();
    } catch (IOException | InterruptedException e) {
        e.printStackTrace();
    }

I set pb.inheritIO() to redirect python prints to standard output. If my python process do some simple things redirection of streams works as expected.

I am using thriftpy (0.3.2) for Java-Python communication. Python process opens a thrift server in a thread:

def __run_server(self, serverport):
    t = Globals.thrifts.split(',')
    thrifts = thriftpy.load(t[0], module_name="processsideserver_thrift", include_dirs=[t[1]])
    self.server = make_server(thrifts.ProcessSideServer, self, '127.0.0.1', serverport)

def run(self):
    if self.can_run == True:
        self.running = True
        try:
            self.server.serve()
        except Exception:
            ''

as self.server.serve() is a blocking function. Because of this line my redirection of streams stopped working. Application and Java-Python communication works fine. But python prints are not showing up in Console.

Thrifpy code is like:

 def serve(self):
    self.trans.listen()
    while not self.closed:
        try:
            client = self.trans.accept()
            t = threading.Thread(target=self.handle, args=(client,))
            t.setDaemon(self.daemon)
            t.start()
        except KeyboardInterrupt:
            raise
        except Exception as x:
            logging.exception(x)

Why it is like that? Because of deamon thread? How can I force redirection of streams anyway?

Just hinting. Can it be related? https://bugs.openjdk.java.net/browse/JDK-8023130 ? And this is another from my own experience: http://stackoverflow.com/questions/3285408/java-processbuilder-resultant-process-hangs — JockX, Nov 18 '15 at 00:38
I test on Linux Mint 17.2. My application doesn't hang out neither. I can do successfully System.out.println in Java method that is invoked from python via thrift. — Marek Jagielski, Nov 18 '15 at 22:58

score 0 · Answer 1 · answered Nov 24 '15 at 01:15

I overloaded the handle thread method of thrifpy TThreadedServer:

def custom_handle(self, client):
    itrans = self.itrans_factory.get_transport(client)
    otrans = self.otrans_factory.get_transport(client)
    iprot = self.iprot_factory.get_protocol(itrans)
    oprot = self.oprot_factory.get_protocol(otrans)
    try:
        while True:
            self.processor.process(iprot, oprot)
            sys.stdout.flush()
            sys.stderr.flush()

    except TTransportException:
        pass
    except Exception as x:
        logging.exception(x)

    itrans.close()
    otrans.close()

I added two lines sys.stdout.flush() and sys.stderr.flush(). It worked. It seems that if we have a while loop there is no time to flush out.

ProcessBuilder : inheritIO not working when python with Thread setDaemon

1 Answers1