Thread-launched running processes won't destroy (Java)

Question

Starting multiple threads and having each exec() then destroy() a running java process result in some of the process not being destroyed and still running after program exit. Here is some code that reproduce the issue. I noticed the more threads you start, the more processes stay alive. And the more sleep before destroy(), the less processes stay alive. (I used InfiniteLoop as an example. Any running process will do the trick.)

EDIT : Bug has been reported to Oracle, waiting for an answer. Feel free to share any knowledge/experiments on the subject.

for(int i = 0; i < 100; i++)
{
  new Thread(new Runnable()
  {
    public void run()
    {
      try
      {
        Process p = Runtime.getRuntime().exec(new String[]{"java", "InfiniteLoop"});
        Thread.sleep(1);
        p.destroy();
      }catch(IOException | InterruptedException e){e.printStackTrace();}                    
    }
  }).start();
}

Answer 1

Use a p.waitFor(); before p.destroy(); ,

this will ensure the completion of the previous process. I think you p.destroy command gets invoked sooner than the exec() command performs the action. Therefore it becomes useless.

Answer 2

If subprocesses write anything to stdout or stderr (intentionally or not), that could cause trouble:

"Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock."

Source: http://www.javaworld.com/jw-12-2000/jw-1229-traps.html

The whole article is IMO worth reading if you need to use Runtime.exec().

Answer 3

You should close the input/output/error streams to the process. We saw some issues in the past where the forked process was not completing properly due to those streams not being closed (even if they weren't being used).

An exemplary solution:

p.destroy();
p.getInputStream().close();
p.getOutputStream().close();
p.getErrorStream().close();

Answer 4

This is simply because before the threads execute the destroy call, your main program terminates and all the associated threads leaving the started processes running. To verify this, simply add a System.out call after the destroy and you will find it is not executed. To overcome this add a Thread.sleep at the end of your main method and you will not have the orphaned processes. The below does not leave any process running.

public class ProcessTest {

public static final void main (String[] args) throws Exception {

    for(int i = 0; i < 100; i++) {
        new Thread(new Runnable()
            {
                public void run() {
                    try {
                        Process p = Runtime.getRuntime().exec(new String[]{"java", "InfiniteLoop"});
                        Thread.sleep(1);
                        p.destroy();
                        System.out.println("Destroyed");
                    }catch(IOException e) {
                        System.err.println("exception: " + e.getMessage());
                    } catch(InterruptedException e){
                        System.err.println("exception: " + e.getMessage());
                    }
                }
            }).start();
    }


    Thread.sleep(1000);

}

}

Answer 5

I believe that according to link, a distinct process is spawned by the operating system in response to this call. This process has a lifetime independent of your Java program and threads within it so you would expect it to continue running after your program has exited. I just tried it on my machine and it appeared to work as expected:

import java.io.*;

class Mp {
public static void main(String []args) {
    for(int i = 0; i < 100; i++) {
        new Thread(new Runnable() {
            public void run() {
                try {
                    System.out.println("1");
                    Process p = Runtime.getRuntime().exec
                        (new String[]{"notepad", ""});
                    System.out.println("2");
                    Thread.sleep(5);
                    System.out.println("3");
                    p.destroy();
                    System.out.println("4");
                }
                catch(IOException | InterruptedException e) {
                    e.printStackTrace();
                }                    
            }
        }).start();
    }
}
}

Answer 6

This is not an answer; I am posting complete source for my own attempt at recreating this problem as per discussion in question comments.

I cannot reproduce this problem on Ubuntu 12.04; OpenJDK 6b_27 (however, see below).

ProcessTest.java:

import java.io.*;

public class ProcessTest {

    public static final void main (String[] args) throws Exception {

        for(int i = 0; i < 100; i++) {
            new Thread(new Runnable() 
                {
                    public void run() {
                        try {
                            Process p = Runtime.getRuntime().exec(new String[]{"java", "InfiniteLoop"});
                            Thread.sleep(1);
                            p.destroy();
                        }catch(IOException e) {
                            System.err.println("exception: " + e.getMessage());
                        } catch(InterruptedException e){
                            System.err.println("exception: " + e.getMessage());
                        }                    
                    }
                }).start();
        }

    }

}

InfiniteLoop.java

public class InfiniteLoop {
    public static final void main (String[] args) {
        while (true) ;
    }
}

I cannot reproduce the issue where processes remaining running after the JVM terminates. However, if I add a long delay in the main thread after starting the threads but before returning from main, I do see roughly a dozen running java processes that stick around (although they are terminated when the main program terminates).

Update:

I just had it leave about 5 processes running after it terminated. It doesn't always happen. Weird. I want to know more about this too. I have a hunch that it has something to do with destroying the process too quickly or some kind of race condition; maybe java forks something off or does something to create a new process that destroy() doesn't take care of if called too quickly / at the wrong time.

I found an old bug (but it is not mark resolved) stating that if a process spawns subprocesses they may not be killed by destroy(). bugs.sun.com/bugdatabase/view_bug.do?bug_id=4770092 What version of the JDK are you using.

Here's another reference to what looks like a similar issue: Java tool/method to force-kill a child process And I want to apologize if I've only added confusion to your life, I don't actually use Process that much and am not familiar with the quirks. Hopefully somebody else will step in with a definitive answer. It seems like it doesn't handle subprocesses well, and I'm presuming java forks something off. That's all I got.

Answer 7

There is a race condition between the time Runtime.exec kicks off a new thread to start a Process and when you tell that process to destroy itself.

I'm on a linux machine so I will use the UNIXProcess.class file to illustrate.

Runtime.exec(...) will create a new ProcessBuilder and start it which on a unix machine creates a new UNIXProcess instance. In the constructor of UNIXProcess there is this block of code which actually executes the process in a background (forked) thread:

java.security.AccessController.doPrivileged(
            new java.security.PrivilegedAction() {
    public Object run() {
    Thread t = new Thread("process reaper") {
        public void run() {
                    try {
                        pid = forkAndExec(prog,
                      argBlock, argc,
                      envBlock, envc,
                      dir,
                      redirectErrorStream,
                      stdin_fd, stdout_fd, stderr_fd);
                    } catch (IOException e) {
                        gate.setException(e); /*remember to rethrow later*/
                        gate.exit();
                        return;
                    }
                    java.security.AccessController.doPrivileged(
                    new java.security.PrivilegedAction() {
                        public Object run() {
                        stdin_stream = new BufferedOutputStream(new
                                                FileOutputStream(stdin_fd));
                        stdout_stream = new BufferedInputStream(new
                                                FileInputStream(stdout_fd));
                        stderr_stream = new FileInputStream(stderr_fd);
                        return null;
                    }
                    });
                    gate.exit(); /* exit from constructor */
        int res = waitForProcessExit(pid);
        synchronized (UNIXProcess.this) {
            hasExited = true;
            exitcode = res;
            UNIXProcess.this.notifyAll();
        }
        }
    };
            t.setDaemon(true);
            t.start();
    return null;
    }
});

Notice that the background thread sets the field pid which is the UNIX process id. This will be used by destroy() to tell the OS which process to kill.

Because there is no way to make sure that this background thread has run when destroy() is called, we may try to kill the process before it has run OR we may try to kill the process before pid field has been set; pid is uninitialized and therefore is 0. So I think calling destroy too early will do the equivalent of a kill -9 0

There is even a comment in the UNIXProcess destroy() that alludes to this but only considers calling destroy after the process has already finished, not before it has started:

// There is a risk that pid will be recycled, causing us to
// kill the wrong process!  So we only terminate processes
// that appear to still be running.  Even with this check,
// there is an unavoidable race condition here, but the window
// is very small, and OSes try hard to not recycle pids too
// soon, so this is quite safe.

The pid field is not even marked as volatile so we may not even see the most recent value all the time.

Answer 8

I had a very similar issue and the problem with destroy() not working was manifesting even with a single thread.

Process process = processBuilder(ForeverRunningMain.class).start()
long endTime = System.currentTimeMillis() + TIMEOUT_MS; 
while (System.currentTimeMillis() < endTime) {
    sleep(50);
}
process.destroy();

The process was not always destroyed if TIMEOUT_MS was too low. Adding an additional sleep() before destroy() fixed it (even though I don't have an explanation why):

Thread.sleep(300);
process.destroy();