4

I have written a service for JIRA(a web application runs in tomcat) which runs periodically(say 1 hour). Basically, the service executes a system command thru runtime.exec(command) and parses the output generated by the command then updates a Lucene index with it, output will be huge.

The problems are:

1) If I shutdown tomcat with shutdown.sh while the above service is executing, the java(or the catalina) process is not getting killed. Both the java & child process are living for a while i.e., until the system command completes & service processes the output. But then the service fails to update the index leaving the index in an inconsistent state.

If I shutdown tomcat when the above service is not running, everything is good. I think, this is explained here. I am still not clear why JVM won't shutdown as the above service is running within tomcat?

Note that this is the only java app running on that machine.

2) Then, if I kill java using kill <pid>, both the java & child process are getting killed contradicting to this post.

Is this because the child process is sending output to parent(java) and once parent is killed, the child has no idea where to send the output and thus got killed ?

3) I tried to use shutdownhook as explained in this post, but that's not working for me. The code inside the shutdownhook is getting executed only after the java & child processes are done with their work. So, calling process.destroy() inside shutdownhook is not useful here.

This seems obvious, as the JVM is still running in my case, it won't call shutdownhooks until it starts it's shutdown sequence. Don't know how this worked for the other guy, I mean, how come the child process spawned by java is still running when JVM is down.

4) If I restart tomcat, new java process with different pid is generated.

Is it possible to stop the child process programmatically when tomcat is shutdown ?

Let me know if I am not clear with my explanation...

Here is the code that executes system command:

        String command = getCommand();

        File view = new File(viewPath);
        Runtime runtime = Runtime.getRuntime();
        try
        {
            final Process process = runtime.exec(command, null, view);

            StreamReader errorStreamReader = new StreamReader(process
                .getErrorStream());
            Thread errorStreamThread = new Thread(errorStreamReader);
            errorStreamThread.start();
            revisions = parseRevisionLogs(process.getInputStream());

            process.waitFor();

            process.getInputStream().close();
            process.getErrorStream().close();
            process.getOutputStream().close();
        }
Community
  • 1
  • 1
ernesto
  • 1,899
  • 4
  • 26
  • 39
  • Can you show how your periodic tasks are created? – Gray Nov 30 '11 at 13:32
  • The above code block will be executed periodically – ernesto Nov 30 '11 at 15:06
  • Which thread is executing that code? The main thread? – Gray Nov 30 '11 at 15:37
  • 1
    So my answer stands then. The reason why your process is not stopping is because the main thread is not a daemon thread and it is reading from your process. You need to run the processing in a daemon thread and be careful if it gets killed while updating the indexes. Or you can do something like @AlexR answer and process a file afterwards. – Gray Nov 30 '11 at 16:16
  • The explanation made me to think abt the actual things happening... Will do some experimenting and come back – ernesto Nov 30 '11 at 17:03

3 Answers3

3

The JVM will not shutdown unless the threads that are left are marked as "daemon". Any non-daemon user threads must finish before the JVM will exit. See this question. If your periodic tasks are not set with setDaemon(true) then they will have to finish before the JVM will exit. You have to call setDaemon before the process starts.

You should be able to make your periodic tasks to be daemon however you do have a race condition with JVM shutdown. You might consider having one daemon task doing the reading from the process but having a non-daemon task do the updating of the index which probably should not get killed while it is working.

Your non-daemon thread could then be sleeping, waiting for the load to finish, and testing to see if it should terminate with a volatile boolean field or other signal.

Community
  • 1
  • 1
Gray
  • 115,027
  • 24
  • 293
  • 354
  • The program will not write anything to indexes until it receives&parses the output. It seems running the system command as daemon process will solve the issue, but I could not figure out how to make the process as daemon, `setDaemon(true)` is for threads ? – ernesto Nov 30 '11 at 15:10
  • Right, you will need to spawn another thread and call `setDaemon(true)` before it is starts. Then your main needs to listen for the tomcat shutdown. – Gray Nov 30 '11 at 17:53
2

I'd suggest you to do the following.

Do not read the process' output directly from java. Instead redirect the output to file and read it from there when process is terminated. Wrap your command using batch file or shell script that stores the PID of separate process, so that you will be able to kill this process separately. Now add shutdown hook to tomcat that will run kill PID where PID is the process ID of separate process.

I believe this will work because now your tomcat and separate process are totally decoupled, so nothing bothers tomcat to shutdown. The same is about the process.

Good luck.

AlexR
  • 114,158
  • 16
  • 130
  • 208
0

Are you doing a waitFor() on the process?

If so you could catch InterruptedException and to a p.destroy()

jontro
  • 10,241
  • 6
  • 46
  • 71