Pentaho | GC overhead limit exceeded

Question

I want to insert data from xlsx file into table. Excel has around 1,20,000 records. But while running transformation, I am getting below error:

GC overhead limit exceeded

I have changed in spoon.bat.

Xmx2g -XX:MaxPermSize=1024m

But still I am getting this error.

Can someone please help on this?

Try bumping Xmx even more and see if that helps. Also it's worth using `ps` command to see if the params you've set actually apply to the process. — Ivan, Oct 16 '16 at 08:36

score 6 · Answer 1 · answered Jun 21 '17 at 16:12

6

In my case , additional to adding the Xms and Xmx parameters(which didnt solve it completly) i added the option -XX:-UseGCOverheadLimit to spoon.sh and the problem solved.

answered Jun 21 '17 at 16:12

Luis Leal

3,388
5
26
49

score 1 · Answer 2 · answered Mar 09 '22 at 06:44

1

Yes it work I increate the memory to 4GB issue is fixed.

if "%PENTAHO_DI_JAVA_OPTIONS%"=="" set PENTAHO_DI_JAVA_OPTIONS="-Xms2048m" "-Xmx4096m"

answered Mar 09 '22 at 06:44

Lenny

11
1

score 0 · Answer 3 · answered Feb 07 '17 at 12:45

This is a known bug with apache POI. the xlsx input step is not able to read big files. I usually turn my files into CVS in such cases. here is the jira case. http://jira.pentaho.com/browse/PDI-5269

The environment variable PENTAHO_DI_JAVA_OPTIONS is used to add option to the starting of jre. Mine is set to "-Xms512m -Xmx3000M -XX:MaxPermSize=256m"

score 0 · Answer 4 · edited Aug 07 '19 at 15:04

0

Split the file into two different files, save half of the data in one file and the remaining in another. It works perfectly.

edited Aug 07 '19 at 15:04

Laurenz Albe

209,280
17
206
263

answered Aug 07 '19 at 14:20

Pravin_Shahapure

11
2

Pentaho | GC overhead limit exceeded

4 Answers4