3

I have a Java application to generate Excel sheets. I am doing it based on the BigGridDemo Example of Apache POI to generate Excel(xlsx).

The idea is to

  1. create a template workbook, create sheets and global objects such as cell styles,number formats etc.
  2. create an application that streams data in a text file
  3. Substitute the sheet in the template with the generated data

In Linux, during the 3rd step, the JVM crashes with this info 

# A fatal error has been detected by the Java Runtime Environment:
#  SIGSEGV (0xb) at pc=0x000000307a772c44, pid=11781, tid=1088649568
#
# JRE version: 6.0_24-b07
# Java VM: Java HotSpot(TM) 64-Bit Server VM (19.1-b02 mixed mode linux-amd64 compressed oops)
# Problematic frame:
# C  [libc.so.6+0x72c44]  memcpy+0x34

The hs_err_pid file has this - 

C  [libc.so.6+0x72c44]  memcpy+0x34

Java frames: (J=compiled Java code, j=interpreted, Vv=VM code)
j  java.util.zip.ZipFile.getNextEntry(JI)J+0
j  java.util.zip.ZipFile.access$400(JI)J+2
j  java.util.zip.ZipFile$2.nextElement()Ljava/util/zip/ZipEntry;+54
j  java.util.zip.ZipFile$2.nextElement()Ljava/lang/Object;+1

Looks like this happens when the template workbook is read as a zip file. This is the code that does this. 

ZipFile zip = new ZipFile(zipfile);
ZipOutputStream zos = new ZipOutputStream(out);

Enumeration<ZipEntry> en = (Enumeration<ZipEntry>) zip.entries();
while (en.hasMoreElements()) {
    ZipEntry ze = en.nextElement();
    if(!ze.getName().equals(entry)){
        zos.putNextEntry(new ZipEntry(ze.getName()));
        InputStream is = zip.getInputStream(ze);
        copyStream(is, zos);
        is.close();
    }
}

How can I avoid this crash?

Wivani
  • 2,036
  • 22
  • 28
kpk
  • 43
  • 1
  • 1
  • 6
  • Did you add a -xMx arg to increase the size of your JVM? Are you running inside something that might have set that argument (Tomcat, Jetty, JBoss)? What is the system memory on the Linux side like when you did this? Are you running low? – chubbsondubs Aug 23 '11 at 14:57
  • Yes. This comes when I tried with -Xmx1024m and 2048 also. I'm not running with App servers. Just as a stand alone java application. There is enough memory in the machine (>15GB). – kpk Aug 23 '11 at 15:09
  • 1
    I'd suggest you switch from BigGridDemo to the new SXSSF. It'll let you do the low memory writing that the BigGridDemo does, but with a much easier API to work with – Gagravarr Aug 23 '11 at 23:16

2 Answers2

1

Given the kind of crash you're having, it appears you are reading and writing to the same zip file.

I wrote up why this happens in another answer, but for your use case, you should write to a different output zip file as you iterate through the input zip file.

Community
  • 1
  • 1
Stuart Caie
  • 2,803
  • 14
  • 15
0

The JVM will crash on linux (possibly other platforms) if you tell it to use 1GB, 2GB, etc and the system doesn't have that much memory available. When the program in the JVM tries to allocate more memory than the max memory setting on the JVM that results in an OutOfMemoryException and the JVM doesn't crash. I'd check and make sure there isn't another program on your system that's eating up more memory than you realize. You can also check the heap dumps that come out of the JVM. When the JVM crashes it writes out a report that tells you more information about the state of the machine when it crashed like how much system memory was being used.

1GB is a massive amount of memory. My IDE doesn't run with that much memory. You probably need to look at what your doing in jconsole or a profiler and see if you're eating up memory yourself. I know that POI can chew up a lot of memory, but you need to figure out a way to pull it back down.

chubbsondubs
  • 37,646
  • 24
  • 106
  • 138
  • I added 1 GB and 2 GB just to make sure that java gets enough memory. It doesnt look like OutOfMemory. This is the memory report during the crash - Heap PSYoungGen total 17984K used 10492K, eden space 15424K 52% used, from space 2560K 53% used, to space 2560K 0% used. There are only 10 rows to generate also. The whole point of using this approach is to reduce the memory consumption of POI. – kpk Aug 23 '11 at 16:04
  • The crash happened inside memcpy so the C code was allocating memory. Typically that means it's trying to increase the heap and the JVM process is under the 1GB limit set by -Xmx. However, if your machine (NOT JVM) is running out of memory the memcpy will fail, and the JVM will crash you as see it. If you look at your program's memory usage it will look normal in these situations. It could crash only using 25M if your linux box is out of memory. – chubbsondubs Aug 23 '11 at 17:15
  • I've also seen crashes when memory stuck in the machine doesn't match. Adding a mixture of 2GB and 1GB modules. Or sometimes the memory timings are mixed and that can cause crashes. – chubbsondubs Aug 23 '11 at 17:17