0

We are working on a performance issue in our application and we observed the following behavior in the thread dump.

There is a thread which owns the montior lock but still in blocked state.

INT:legacon:41:76" prio=10 tid=0x0000000055df0800 nid=0x2e39 waiting for monitor entry     [0x00002aaacb4d4000]
   java.lang.Thread.State: BLOCKED (on object monitor)
    at com.leg.common.connections.fg.FgConnection.send(FgConnection.java:342)
    - locked <0x00000007345b78c0> (a java.io.BufferedOutputStream)
    at com.leg.mg.core.po.storage.ReplicationConnection.send(ReplicationConnection.java:190)
    at com.leg.mg.core.po.storage.ClusterStorageManager.replicate(ClusterStorageManager.java:620)
    at com.leg.mg.core.po.poEngine.store(poEngine.java:1117)
    at com.leg.mg.core.po.poWorker.storeMessage(poWorker.java:1024)
    at com.leg.mg.proto.RWorker$DocumentMessageSender.run(RouterWorker.java:1976)
    at com.leg.common.threadpool.VirtualThreadGroup$VirtualThreadGroupRunnable.run(VirtualThreadGroup.java:147)
    at com.leg.common.threadpool.ThreadControl.run(ThreadControl.java:201)
    at com.leg.common.threadpool.ThreadPool$PoolThread.run(ThreadPool.java:742)

The corresponding code block is

line 250  out = new BufferedOutputStream(socket.getOutputStream(), BUFFER_LENGTH);

line 331  public void send(Object obj, boolean flush) throws ConnectionException {
.
.
line 342   synchronized(out) {
.         if(!isOpen()) {
.           throw new ConnectionException("Connection closed", url);
.         }  
.
.         header.reset();
.         header.write(B_SEND_EX);
.         header.write((byte)'\r');
.         header.write((byte)'\n');
.         int len = splitInt(val.length);
.         while (len-- > 0) {
.           header.write(splitBuffer[len]);
.         }
.         header.write((byte)'\r');
.         header.write((byte)'\n');
.         header.writeTo(out);
.            out.write(val);
.        if (flush) {
.        out.flush();
.        }
.          }
line 364}

As per the above stack trace, it owns the montior lock on "out" at line 342. But it is still in BLOCKED state. Aslo, it says waiting for monitor entry [0x00002aaacb4d4000]. We could not indentify which is that 0x00002aaacb4d4000 and who is owning the lock on it, because there is no other occurrence for 0x00002aaacb4d4000 except this place in the whole thread dump.

How the thread will be in blocking state even after owning monitor lock?

How to identify what is 0x00002aaacb4d4000 ? Will it be any internal java object ?

Java version: jdk1.6.0_32

It will be really great if you can help us to understand this case.

vijayashankard
  • 721
  • 1
  • 7
  • 23
  • 2
    The monitor it's waiting on is not the one it owns. – user253751 May 27 '14 at 05:04
  • Looks like system resources used by `out` is in the locked state. If `out` isn't the only accessor to the socket connection, synchronizing with `out` is might not sufficient. – 9dan May 27 '14 at 05:12
  • @immibis what would be it is waiting for? I couldnt find anyother thread owns lock on 0x00002aaacb4d4000. Actually there is no other occurrence for 0x00002aaacb4d4000 in the entire dump except this place – vijayashankard May 27 '14 at 05:31
  • @9dan 'out' is the only accessor to the socket. Is there a way to confirm or relate the object 0x00002aaacb4d4000 to the system resource. – vijayashankard May 27 '14 at 05:33
  • Does anything else in your code synchronize on the socket? I noticed that `out` is a BufferedOutputStream wrapper, not the actual socket stream. – user253751 May 27 '14 at 06:13
  • Also I don't think [0x00002aaacb4d4000] is the monitor it's waiting for. According to some random examples I found via Google, there should be another line like `- waiting to lock <0x0000000780a000b0> (a com.nbp.theplatform.threaddump.ThreadBlockedState)` which shows the monitor it's waiting for. – user253751 May 27 '14 at 06:18
  • This question is simillar to http://stackoverflow.com/questions/20382091/thread-dump-blocked-and-locked. – vijayashankard May 27 '14 at 06:58
  • @immibis In the above link, one of the answer relates it to this [BUG](https://bugs.openjdk.java.net/browse/JDK-8036823). But as per this bug there should be two threads entries with "locked" for the same object. But in my case, only this thread locks this object (0x00000007345b78c0). Still not sure which is that 0x00002aaacb4d4000. I dont have have any other synchronization on the socket also. – vijayashankard May 27 '14 at 07:04

0 Answers0