0

I am testing a simple recommender system built using Mahout in Eclipse.

If the preferences file is too big I am getting this error:

    Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.lang.String.substring(String.java:1939)
    at java.lang.String.subSequence(String.java:1972)
    at com.google.common.base.Splitter$SplittingIterator.computeNext(Splitter.java:612)
    at com.google.common.base.Splitter$SplittingIterator.computeNext(Splitter.java:522)
    at com.google.common.base.AbstractIterator.tryToComputeNext(AbstractIterator.java:64)
    at com.google.common.base.AbstractIterator.hasNext(AbstractIterator.java:59)
    at com.google.common.base.AbstractIterator.next(AbstractIterator.java:74)
    at org.apache.mahout.cf.taste.impl.model.file.FileDataModel.processLine(FileDataModel.java:408)
    at org.apache.mahout.cf.taste.impl.model.file.FileDataModel.processFile(FileDataModel.java:363)
    at org.apache.mahout.cf.taste.impl.model.file.FileDataModel.buildModel(FileDataModel.java:259)
    at org.apache.mahout.cf.taste.impl.model.file.FileDataModel.reload(FileDataModel.java:231)
    at org.apache.mahout.cf.taste.impl.model.file.FileDataModel.<init>(FileDataModel.java:221)
    at org.apache.mahout.cf.taste.impl.model.file.FileDataModel.<init>(FileDataModel.java:169)
    at org.apache.mahout.cf.taste.impl.model.file.FileDataModel.<init>(FileDataModel.java:149)
    at com.predictionmarketing.itemrecommend.ItemRecommend.main(ItemRecommend.java:35)

The file contains about 5 millions preference points.

I have already read several messages about this issue and my eclipse.ini is the following

-startup
../../../plugins/org.eclipse.equinox.launcher_1.3.0.v20140415-2008.jar
--launcher.library
../../../plugins/org.eclipse.equinox.launcher.cocoa.macosx.x86_64_1.1.200.v20140603-1326
-product
org.eclipse.epp.package.java.product
--launcher.defaultAction
openFile
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
3000m
--launcher.defaultAction
openFile
--launcher.appendVmargs
-vmargs
-Dosgi.requiredJavaVersion=1.6
-XstartOnFirstThread
-Dorg.eclipse.swt.internal.carbon.smallFonts
-XX:MaxPermSize=3000m
-Xms3000m
-Xmx6000m
-Xdock:icon=../Resources/Eclipse.icns
-XstartOnFirstThread
-Dorg.eclipse.swt.internal.carbon.smallFonts

Basically I added -Xmx6000m and changed -Xmx and MaxPermSize (both) to 3000m. I am not very fluent in Java so I don't know if the settings are correct, but during the execution I can see something like

Heap size: 55M of total 3017M max 5996 mark 55M

Which seems to reflect the settings.

The weird thing is that I don't see much increase of the heap size during execution, also the free memory in my computer (mac os) is around 800M.

Where is the problem? If the preferences file is smaller, everything works fine.

Eugenio
  • 3,195
  • 5
  • 33
  • 49
  • Is this a program you are running from within Eclipse with the Run command? You need to change the settings in 'Run > Run Configurations' for those. – greg-449 Feb 10 '15 at 17:39
  • You can try using file mapping. Here's a useful link: http://www.javacodegeeks.com/2013/05/power-of-java-memorymapped-file.html –  Feb 10 '15 at 17:56
  • @greg-449 yes I am running from Eclipse. I have opened Run > Run Configurations but which are the settings I nee to change? – Eugenio Feb 10 '15 at 18:51

2 Answers2

3

The eclipse.ini settings are only used for the Eclipse application.

When you Run a Java application from Eclipse you need to go to 'Run > Run Configurations'. Find your application in the Java application list. On the 'Arguments' tab enter your memory settings in the 'VM arguments' section.

greg-449
  • 109,219
  • 232
  • 102
  • 145
  • I added -Xms3000m -Xmx6000m in the VM arguments section and it works perfectly, thanks! Does Ecplise also needs, consequently, the same parameters (in eclipse.ini)? – Eugenio Feb 11 '15 at 10:17
  • 1
    No, when you Run a program a new JVM is started so it does not change the memory required by Eclipse. – greg-449 Feb 11 '15 at 10:21
0

Read Java Heap Space - How does -Xmx work exactly?

and

http://javarevisited.blogspot.gr/2011/05/java-heap-space-memory-size-jvm.html

Java Heap is not the same for every system i think that in Windows for example the default java Heap is 60 Mb

Community
  • 1
  • 1
crAlexander
  • 376
  • 1
  • 2
  • 12