0

I have a service to build a report in JAVA that are using Apache POI, but at this moment i have a problem.

I need allow the report generation with a lot of data (2 millions of rows) and only the map creation with that data spends 4 GB of memory more the memory that spends apache POI to build the report, i need more than 8 GB.

This process is launched manually for somebody, but the person can run the process whenever he wants.

This process will be executed only two or three times per week, then I don't know if having a machine with a lot of memory is the solution, because they are resources that are being spent only for 2 or 3 executions per week.

Or I don't know if exists a tool to help with the memory on demand, like redis or How can I attack this problem without spending so many resources?

Code Geas Coder
  • 1,839
  • 4
  • 23
  • 29
  • Don't store the whole 2 million rows in memory. Write/print the pages to file as you go, that way you really only need to keep one or two pages worth of data in memory at a time, and you should be nowhere near your memory cap. Apache has the ability to only read select rows at a time, see here for example: [How to load a large xlsx file with Apache POI?](https://stackoverflow.com/questions/11891851/how-to-load-a-large-xlsx-file-with-apache-poi) Or for even more fine control of the sheet data see: https://poi.apache.org/components/spreadsheet/how-to.html#xssf_sax_api – sorifiend Jul 08 '22 at 04:25
  • If you need to process all 2 million rows and do some calculations, then this can still be done a few rows at a time for example by stripping out only the essential data – sorifiend Jul 08 '22 at 04:29

0 Answers0