66

How can I programmatically detect that a deadlock has occurred in a Java program?

oxbow_lakes
  • 133,303
  • 56
  • 317
  • 449
  • 1
    You should specify whether you need to detect this programmatically or via some kind of external monitoring tool? – oxbow_lakes Jul 09 '09 at 07:49
  • Ah, my answer was assuming using a tool. I guess it is useful to know about both approaches anyway. – RichardOD Jul 09 '09 at 11:17
  • 1
    The JavaSpecialist website (always worth reading) has an [interesting article](http://www.javaspecialists.eu/archive/Issue130.html) on this, discussing the theory and practice. – Brian Agnew Jul 09 '09 at 09:20

9 Answers9

69

You can do this programmatically using the ThreadMXBean that ships with the JDK:

ThreadMXBean bean = ManagementFactory.getThreadMXBean();
long[] threadIds = bean.findDeadlockedThreads(); // Returns null if no threads are deadlocked.

if (threadIds != null) {
    ThreadInfo[] infos = bean.getThreadInfo(threadIds);

    for (ThreadInfo info : infos) {
        StackTraceElement[] stack = info.getStackTrace();
        // Log or store stack trace information.
    }
}

Obviously you should try to isolate whichever thread is performing this deadlock check - Otherwise if that thread deadlocks it won't be able to run the check!

Incidentally this is what JConsole is using under the covers.

Adamski
  • 54,009
  • 15
  • 113
  • 152
  • 4
    The Javadoc for that method says it might be expensive. Do you happen to know about how expensive? – Bart van Heukelom May 04 '11 at 10:46
  • 1
    No I don't although I guess you could write some code to time the operation and see how this scales as thread count / number of locks increases. – Adamski Jul 21 '14 at 20:49
  • There are a few problems with this code. First, findDeadlockedThreads could return null for some threads which can cause NPE in your loop. Second, this will not print actual stack trace, you need to get actual threads from thread info. You can check this post for more info: http://korhner.github.io/java/multithreading/detect-java-deadlocks-programmatically/ – korhner Feb 17 '15 at 20:21
  • @korhner: findDeadlockedThreads() will return null in the case of there being no deadlocked threads, which I already check for; I don't believe it ever returns an array containing sporadic null values. getThreadInfo can return an array containing nulls if the corresponding threads are not alive. However, I already know they are alive (and deadlocked) from the previous call. Re. the second point I've left stack trace logging as an exercise for the OP but they don't actually mention that they want to do this. – Adamski Feb 18 '15 at 11:35
  • @Adamski:findDeadlockedThreads can report deadlocks even if a thread has a timeout on a lock which in practice is not a real dedlock (after timeout lock is released). In theory it can happen that the thread is destroyed after timeout and before your getThreadInfo. You are right about findDeadlockedThreads won't return null for specific threads however, sorry. – korhner Feb 18 '15 at 12:24
  • It is working for me in some cases. But not for all cases. – Amit Kumar Gupta Sep 11 '16 at 16:00
  • I benchmarked the findDeadlockedThreads method (on a HotSpot Server VM with only few threads, admittedly). Each call takes less than 100 µs after warmup. http://tinybrain.de/1009294 – Stefan Reich Jul 21 '17 at 14:49
  • Benchmark results for more threads: For 100 sleeping threads the detection takes <250 µs. For 1000 sleeping threads it takes <4 ms. Still very OK to do occasionally. – Stefan Reich Jul 21 '17 at 15:03
  • In case you wondered why stack trace is empty in above example: Use getThreadInfo(threadIds, Integer.MAX_VALUE) to acquire all stack trace elements. – Henrik Steudel Sep 06 '17 at 10:51
14

One useful hint for investigation:

If you can catch the application red handed and suspect a deadlock has occurred, go and press "Ctrl-Break" in the java.exe console window (or "Ctrl-\" on Solaris/Linux). The jvm will dump the current status and stack trace of all threads, find out dead locks and precisely describe them.

It will look something like this:

Full thread dump Java HotSpot(TM) Client VM (1.5.0_09-b03 mixed mode):

"[Test Timer] Request Queue" prio=6 tid=0x13d708d0 nid=0x1ec in Object.
    wait() [0x1b00f000..0x1b00fb68]
    at java.lang.Object.wait(Native Method)
    at java.lang.Object.wait(Unknown Source)
    at library.util.AsyncQueue.run(AsyncQueue.java:138)
        - locked <0x02e70000> (a test.server.scheduler.SchedulerRequestQueue)

    ...

Found one Java-level deadlock:
=============================
"Corba service":
  waiting to lock monitor 0x13c06684 (object 0x04697d90, a java.lang.Object),
  which is held by "[Server Connection] Heartbeat Timer"
"[Server Connection] Heartbeat Timer":
  waiting to lock monitor 0x13c065c4 (object 0x0467e728, a test.proxy.ServerProxy), which is held by "Corba service"

Java stack information for the threads listed above:
===================================================
"Corba service":
    at test.proxy.ServerProxy.stopHBWatchDog(ServerProxy:695)
    - waiting to lock <0x04697d90> (a java.lang.Object)
    ...
Nicolas Simonet
  • 533
  • 4
  • 10
  • 1
    Or ctrl-\ from Solaris/Linux. – Tom Hawtin - tackline Jul 09 '09 at 11:48
  • 2
    Actually what you want is send a "QUIT" signal (ie a SIGQUIT) but it's not really a 'QUIT'. A SIGQUIT on OS X, Linux (and Solaris) on a Java app dumps the stacktrace. Ctrl+\ is one way to send a SIGQUIT. Taking the pid and doing: *kill -3 {id}* is another way to do it. – SyntaxT3rr0r Mar 15 '10 at 14:29
6

You can detect the deadlocked threads programmatically using ThreadMXBean class.Here is the code,

    ThreadMXBean bean = ManagementFactory.getThreadMXBean();

    long ids[] = bean.findMonitorDeadlockedThreads();

    if(ids != null)
    {
        ThreadInfo threadInfo[] = bean.getThreadInfo(ids);

        for (ThreadInfo threadInfo1 : threadInfo)
        {
            System.out.println(threadInfo1.getThreadId());    //Prints the ID of deadlocked thread

            System.out.println(threadInfo1.getThreadName());  //Prints the name of deadlocked thread

            System.out.println(threadInfo1.getLockName());    //Prints the string representation of an object for which thread has entered into deadlock.

            System.out.println(threadInfo1.getLockOwnerId());  //Prints the ID of thread which currently owns the object lock

            System.out.println(threadInfo1.getLockOwnerName());  //Prints name of the thread which currently owns the object lock.
        }
    }
    else
    {
        System.out.println("No Deadlocked Threads");
    }

Click here for more info on how to detect the deadlocked threads.

user2485429
  • 567
  • 8
  • 7
  • Better to use findDeadlockedThreads() instead of the findMonitorDeadlockedThreads(). The findDeadlockedThreads has been introduced on 1.6 and is better in comparison with the findMonitorDeadlockedThreads() that may loose some deadlock cases and not report them. It is not able to report deadlocks for: ReentrankLock or ReentrantReadWriteLock. More info can be found here: https://meteatamel.wordpress.com/2012/03/21/deadlock-detection-in-java/ under the "findMonitorDeadlockedThreads vs. findDeadlockedThreads" title – Investigator Feb 13 '19 at 12:09
3

JArmus is a library for deadlock detection and avoidance. It includes support for: Thread.join, CyclicBarrier, CountDownLatch, Phaser, and ReentrantLock.

To use JArmus you need to instrument your code. Either through one of its instrumented classes or automatically with the JArmus instrumentar jarmusc.

java -jar jarmusc.jar yourprogram.jar checkedprogram.jar

The input yourprogram.jar is the program you want to check. The output is the same program with checks to automatically find any deadlock.

Barriers need some help

Verifying deadlocks with classes CyclicBarrier, CountDownLatch, Phaser is a bit tricky --- for example, JConsole cannot detect these types of deadlocks. JArmus needs a little help from you: you must specify which threads are influencing synchronization, we call these registered threads.

As soon as possible, the thread must mark itself as registered. A good place to mark registered threads is at the beginning method Runnable.run. JArmus.register(latch);

Example

The following program that deadlocks is correctly identified by JArmus:

final CountDownLatch latch = new CountDownLatch(2);
final CyclicBarrier barrier = new CyclicBarrier(2);
final Queue<Exception> exceptions = new ArrayDeque<>();
Thread t1 = new Thread(new Runnable() {
    @Override
    public void run() {
        try {
            JArmus.register(barrier); // do not forget to register!
            JArmus.register(latch); // do not forget to register!
            latch.countDown();
            latch.await();
            barrier.await();
        } catch (Exception e) {
            exceptions.add(e);
        }
    }
});
Thread t2 = new Thread(new Runnable() {
    @Override
    public void run() {
        try {
            JArmus.register(barrier); // do not forget to register!
            JArmus.register(latch); // do not forget to register!
            barrier.await();
            latch.countDown();
            latch.await();
        } catch (Exception e) {
            exceptions.add(e);
        }
    }
});
t1.start();
t2.start();
2

You might want to consider IBM's MTRAT. Prevention is better than cure after all. The Multicore Software Development Kit also comes with a deadlock detection tool.

Moffee
  • 401
  • 5
  • 15
RichardOD
  • 28,883
  • 9
  • 61
  • 81
2

If you don't require programmatic detection you can do this via the JConsole; on the thread tab there is a "detect deadlock" button. In JDK6 this detect locks for both intrinsic monitors and j.u.c Locks

Run up the JConsole via the $JAVA_HOM/bin/jconsole command

oxbow_lakes
  • 133,303
  • 56
  • 317
  • 449
1

tempus-fugit also implements it along with a programmatic thread dumping class. It's implemented using the mbean mechanism mentioned above and offers a drop in, out-of-the-box super duper solution.

Toby
  • 9,523
  • 8
  • 36
  • 59
1

There is code here: http://www.java2s.com/Code/Java/Development-Class/PerformingdeadlockdetectionprogrammaticallywithintheapplicationusingthejavalangmanagementAPI.htm

The magic happens in ThreadMonitor.findDeadlock():

  public boolean findDeadlock() {
    long[] tids;
    if (findDeadlocksMethodName.equals("findDeadlockedThreads")
        && tmbean.isSynchronizerUsageSupported()) {
      tids = tmbean.findDeadlockedThreads();
      if (tids == null) {
        return false;
      }

      System.out.println("Deadlock found :-");
      ThreadInfo[] infos = tmbean.getThreadInfo(tids, true, true);
      for (ThreadInfo ti : infos) {
        printThreadInfo(ti);
        printLockInfo(ti.getLockedSynchronizers());
        System.out.println();
      }
    } else {
      tids = tmbean.findMonitorDeadlockedThreads();
      if (tids == null) {
        return false;
      }
      ThreadInfo[] infos = tmbean.getThreadInfo(tids, Integer.MAX_VALUE);
      for (ThreadInfo ti : infos) {
        // print thread information
        printThreadInfo(ti);
      }
    }

    return true;
  }

This calls an API of the ThreadMXBean which has a different name in Java 5 and 6 (hence the outer if()).

The code example also allows to interrupt the locks, so you can even break the deadlock.

Aaron Digulla
  • 321,842
  • 108
  • 597
  • 820
0

In case you want it to be done in run-time you can use watchdog for that.

Artem Barger
  • 40,769
  • 9
  • 59
  • 81