3

I am reading an Excel sheet into R with XLConnect. It works very well. However, if I re-run the command (after changing values in the Excel file, for example), the function runs out of memory.

The file/sheet I am reading has 18 columns and 363 rows of numeric data.

The error message is

Error: OutOfMemoryError (Java): Java heap space

which appears on the second (identical) run of a readWorksheetFromFile call. I am trying to produce an MWE by repeatedly running the input call from this example, but the error does not seem to be reproducible with that file.

The Excel file I am using has many interconnected sheets and is about 3 MB. The sheet that I am reading is also linked to others, but I have set useCachedValues = TRUE.

It seems to me that, after executing the first call, the Java memory is not cleared. The second call then attempts to fill more data into memory, which causes the call to fail. Is it possible to force a garbage collection on the Java memory? Currently, the only solution is restarting the R session, which is not practical for my clients.

I know that expanding the Java memory might solve this, but that strikes me as a clumsy solution. I would prefer to find a way to dump the memory from previous calls.

I have also tried using the more verbose loadWorkbook and readWorksheet functions. The same error occurs.

Let me know if there is any other useful information you may require!

dynamo
  • 2,988
  • 5
  • 27
  • 35
  • You can see this post http://stackoverflow.com/questions/14763079/what-are-the-xms-and-xmx-parameters-when-starting-jvms – jrey Nov 06 '13 at 12:50

1 Answers1

2

You should have a look at 

?xlcFreeMemory

and 

?xlcMemoryReport

which is also mentioned in the XLConnect package docu if you are having multipe runs and want to clean up in between.

GWD
  • 1,387
  • 10
  • 22
  • Great, thanks. I've wrapped the ``readWorksheetFromFile`` in a convenience function, which seems to have alleviated the issue (as long as the wrapper function exists without error). But the things you suggest seem to provide the solution to the underlying problem. – dynamo Nov 07 '13 at 12:16