1

I am facing an issue when trying to open and read an xlsx file size of around 100MB, 650k rows and about a total of 10,400,000 cells count.

This is the issue i get:

org.apache.poi.util.RecordFormatException: Tried to read data but the maximum length for this record type is 100,000,000.
If the file is not corrupt and not large, please open an issue on bugzilla to request 
increasing the maximum allowable size for this record type.
You can set a higher override value with IOUtils.setByteArrayMaxOverride()

It hit error on this code:

FileInputStream fis = new FileInputStream(file);
XSSFWorkbook newWorkbook = new XSSFWorkbook(fis);

I tried using InputStream instead of FileInputStream, same issue. And if i add

IOUtils.setByteArrayMaxOverride(1000000000);

I get the java heap space error:

Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space

My eclipse.ini settings already at max 2GB heap space (-Xms256m -Xmx2048m). Are there anything that i can do to read more than the maximum length ?

poojay
  • 117
  • 1
  • 2
  • 9
  • Try run parameter -Xmx?g, where ? = number of gigabytes of maximum heap size. – szeak Nov 21 '22 at 10:00
  • @szeak my eclipse.ini is alreayd at max limit. -Xms256m -Xmx2048m – poojay Nov 21 '22 at 10:05
  • I was thinking put this in `Run->Run...->VM arguments` section. – szeak Nov 21 '22 at 10:18
  • It is more memory friendly, when you thinking in other way of processing the data. e.g. convert it to CSV, than read and process the csv line-by-line. – szeak Nov 21 '22 at 10:26
  • @szeak i added the arguments, still facing the same issue. i added -Xms256m -Xmx8g – poojay Nov 21 '22 at 10:28
  • @szeak dont think i can convert to CSV, because i need to open the excel first, which i having issue on that line – poojay Nov 21 '22 at 10:29
  • We have a good answer here: [Processing large xlsx file](https://stackoverflow.com/questions/4897766/processing-large-xlsx-file) – szeak Nov 21 '22 at 10:36
  • Does this answer your question? [Processing large xlsx file](https://stackoverflow.com/questions/4897766/processing-large-xlsx-file) – szeak Nov 21 '22 at 10:37
  • Note: The settings in the eclipse.ini are just for Eclipse itself, they are nothing to do with programs you run from Eclipse - those are set in the Run Configuration. – greg-449 Nov 21 '22 at 10:50

0 Answers0