22

I am running a Middleman (which uses Webrick) server on JRuby inside a JVM process using the org.jruby.embed.ScriptingContainer API.

If I shutdown cleanly and stop the server from inside the JVM, everything works as expected.

But if I send a SIGTERM to the JVM process (for example, by hitting ctrl+C at the command line), the console returns but the JVM process does not terminate - it hangs around indefinitely until I send it a SIGKILL.

I tried registering a JVM shutdown hook to terminate the ScriptingContainer instance, but the hook never fires. I'm not sure why... perhaps JRuby is swallowing the SIGTERM somehow?

How can I get the JVM to shut all the way down, cleanly, even if it contains a running Webrick server?

levand
  • 8,440
  • 3
  • 41
  • 54
  • Have you verified that the problem is JRuby, perhaps by swapping `ScriptingContainer` for a does-nothing implementation of [`org.jruby.embed.EmbedRubyInstanceConfigAdapter`](http://javadox.com/org.jruby/jruby-complete/1.7.13/org/jruby/embed/EmbedRubyInstanceConfigAdapter.html) and seeing if the problem persists? – durron597 Apr 27 '15 at 18:49
  • 10
    Control-C doesn't send SIGTERM, it sends SIGINT – rghome Apr 27 '15 at 19:31
  • 2
    What OS are you on? Could you provide a thread dump of the JVM after you've hit ctrl+c? (run `jstack `) How do you check if your shutdown hook fires? – K Erlandsson Apr 29 '15 at 21:05
  • Following [WEBrick documentation](http://ruby-doc.org/stdlib-1.9.3/libdoc/webrick/rdoc/WEBrick.html) I read it as WEBrick is able to install a SIGINT handler. So Middleman would do this part. Maybe the source of your problem is at a different place. Have you tried to connect to the session for example with `jconsole` to check which thread is still alive. – SubOptimal May 04 '15 at 10:42
  • Are you creating any threads, or does the ScriptingContainer create any? – schtever Jul 24 '15 at 02:58

2 Answers2

1

Looks like kill -9 does not trigger the shutdownHook, but kill -15 does.

This StackOverflow question has a detailed answer to the same problem.

Bottom line is, you're screwed, as there doesn't seem to exist any way to intercept a kill -9 signal and perform some maintenance tasks before the JVM halts.

Community
  • 1
  • 1
Olivier Croisier
  • 6,139
  • 25
  • 34
  • signal man page ""The signals SIGKILL and SIGSTOP cannot be caught, blocked, or ignored.""", you can't really handle sigkill even in C program. http://man7.org/linux/man-pages/man7/signal.7.html – davide May 17 '15 at 10:30
  • I don't need to handle SIGKILL... That is fine as it still results in a complete process shutdown. My problem is detecting and terminating on a SIGTERM or SIGINT. – levand May 18 '15 at 19:55
  • SIGINT and SIGTERM signals were observed on Linux but in Windows it never sent these signals to the application. – Kiran Sep 29 '15 at 10:07
0

First, a debugging trick: you can send a SIGQUIT to a Java process to get it to print the current stack traces of all running threads. This can help you diagnose which threads are causing the JVM to not exit.

Sending a SIGINT, SIGHUP, or SIGTERM causes Java to run its shutdown hooks, then exit. You can see this in java.lang.Terminator. Once the signal is received, java.lang.Shutdown handles running all the shutdown hooks before terminating the process. Shutdown.halt() is not called until after the shutdown hooks have all finished, which suggests you have a shutdown hook that is hanging.

Without actual code or stack traces it's difficult to provide a more precise answer than that, but a JVM that stays alive after you send a SIGINT is usually doing something strange in its shutdown hooks. If this isn't enough to go on try sending a SIGINT, waiting a few seconds, then sending a SIGQUIT, and adding the resulting stack traces to the question.

dimo414
  • 47,227
  • 18
  • 148
  • 244