10

We are working over our client's production server heap to detect and solve memory leaks. For this we are using jmap periodically to collect the necessary information.

But last week we couldn't take the dump, because it triggered a EOF error and shutdown the Tomcat instance.

I searched on the internet but couldn't find any concrete information about this error. We detected that it only occurs when using the Gc First garbage collector algorithm.

This is the command line we used to perform the jmap:

jmap -dump:format=b,file=heap.bin <PID>

Java version on the server: JDK 1.7.0_7 x64

Has anyone already faced this kind of error? Maybe some configuration that is missing or a patch to java/jmap required.

UPDATE

A few more information that we have collected about this error:

[root]# jmap -dump:format=b,file=heap.bin 7806
    Dumping heap to /tmp/heap.bin ...
    Exception in thread "main" java.io.IOException: Premature EOF
        at sun.tools.attach.HotSpotVirtualMachine.readInt(HotSpotVirtualMachine.java:244)
        at sun.tools.attach.LinuxVirtualMachine.execute(LinuxVirtualMachine.java:193)
        at sun.tools.attach.HotSpotVirtualMachine.executeCommand(HotSpotVirtualMachine.java:213)
        at sun.tools.attach.HotSpotVirtualMachine.dumpHeap(HotSpotVirtualMachine.java:180)
        at sun.tools.jmap.JMap.dump(JMap.java:241)
        at sun.tools.jmap.JMap.main(JMap.java:140)
[root]#

Note: the target directory has over 500gb of free space

Error outputed to the catalina.out (JVM dump error):

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f0269cc41c6, pid=7806, tid=139647231129360
#
# JRE version: Java(TM) SE Runtime Environment (7.0_40-b43) (build 1.7.0_40-b43)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (24.0-b56 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# V  [libjvm.so+0x58c1c6]  DumperSupport::dump_field_value(DumpWriter*, char, unsigned char*)+0x1c6
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /opt/tomcat6/bin/hs_err_pid7806.log
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
vkrausser
  • 393
  • 1
  • 3
  • 17
  • I am just facing the same problem with JDK 1.7.0_45. Most heap dumps crash the VM an dump a core with the same problematic frame that you mentioned. This makes analysing memory leaks in java impossible. Did you find any solution to the problem? The only thing I can think of now is to switch back to CMS / other GC. I reported it with Bug ID: 900949 – mnp Jan 09 '14 at 10:52
  • I still don't have any solution to these problem @mnp. I've also reported this in december with Bug ID 9008971, without any response yet. – vkrausser Jan 13 '14 at 18:55
  • Hello, I can't find either of your two bugs in the oracle bug db, the only related bug I could see is resolved http://bugs.java.com/bugdatabase/view_bug.do?bug_id=6882554 I can however confirm that this is still happening on 1.7u45 with G1 – DrGranit Mar 14 '14 at 13:17
  • Hello @DrGranit! I'm also not able to find the bug report in the oracle bug db. It's really strange, but I have followed all the instructions on the JVM dump and fulfilled the bug report submission. I really don't know where these information goes, I just hope that someboy reads it :) – vkrausser Mar 28 '14 at 12:14
  • 1
    Still happening with jdk 1.7u55. – mnp Jun 16 '14 at 11:34
  • Maybe we should open a new case? – adosaiguas Jul 11 '14 at 08:02
  • This still happens with JDK 1.7u65. Using option "-F" the dump can be generated successfully, but it takes forever so that's not really a solution for larger heaps. – Jose Otavio Aug 12 '14 at 13:06
  • Any updates on that? The issue still persists in java **1.8.0_45** – jdevelop May 20 '15 at 17:32
  • It sounds like you should all file bugs against the JDK. – dimo414 Jun 13 '15 at 17:28
  • I'm voting to close this question as off-topic because it's a Java bug report, not a question. – dimo414 Jun 13 '15 at 17:29
  • @dimo414 a bug report was filed at the time. When I made the question the idea was to know if the problem only happend with me or if it happened because of some mistake I made. I agree that at its point its probably a real bug and it does not fit in the Stackoverflow format. – vkrausser Jun 15 '15 at 11:52
  • @vkrausser do you have a link to the bug you filed? It would be helpful for other people who find this, I'm sure. – dimo414 Jun 16 '15 at 00:09
  • @dimo414 when I submited I recived the ID 9008971 in a e-mail. But I can find it on the bug database at http://bugs.sun.com. This email was send to me on 12-16-2013 – vkrausser Jun 17 '15 at 13:55
  • @vkrausser according to [the FAQ](http://bugs.java.com/faq.do) "Not all bugs submitted do end up in the database." perhaps they didn't have enough information from your initial report? If it's repeatable, an [SSCCE](http://sscce.org/) might help them. – dimo414 Jun 17 '15 at 18:26
  • Is there any resolution to this issue? – Steephen Dec 18 '15 at 02:27

1 Answers1

3

I could able to resolve the issue by using a few additional options.

java version "1.7.0_45".

Java process was configured with first garbage collection algorithm: -XX:+UseG1GC 

jmap -J-d64 -dump:live,format=b,file=<heap_dump_filename> <PID>
Steephen
  • 14,645
  • 7
  • 40
  • 47
  • Please let me know the exact command to add java version in jmap command. I did not find that option in internet. I have same issue @ http://stackoverflow.com/questions/36604102/java-heap-dump-error-with-jmap-command-premature-eof – Ravindra babu Jul 05 '16 at 16:00
  • You don't need to pass java version as a parameter with jmap command. You can find java version using `java -version` command though!! – Steephen Jul 05 '16 at 20:53