How to solve "java.io.IOException: error=12, Cannot allocate memory" calling Runtime#exec()?

Question

On my system I can't run a simple Java application that start a process. I don't know how to solve.

Could you give me some hints how to solve?

The program is:

[root@newton sisma-acquirer]# cat prova.java
import java.io.IOException;

public class prova {

   public static void main(String[] args) throws IOException {
        Runtime.getRuntime().exec("ls");
    }

}

The result is:

[root@newton sisma-acquirer]# javac prova.java && java -cp . prova
Exception in thread "main" java.io.IOException: Cannot run program "ls": java.io.IOException: error=12, Cannot allocate memory
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:474)
        at java.lang.Runtime.exec(Runtime.java:610)
        at java.lang.Runtime.exec(Runtime.java:448)
        at java.lang.Runtime.exec(Runtime.java:345)
        at prova.main(prova.java:6)
Caused by: java.io.IOException: java.io.IOException: error=12, Cannot allocate memory
        at java.lang.UNIXProcess.<init>(UNIXProcess.java:164)
        at java.lang.ProcessImpl.start(ProcessImpl.java:81)
        at java.lang.ProcessBuilder.start(ProcessBuilder.java:467)
        ... 4 more

Configuration of the system:

[root@newton sisma-acquirer]# java -version
java version "1.6.0_0"
OpenJDK Runtime Environment (IcedTea6 1.5) (fedora-18.b16.fc10-i386)
OpenJDK Client VM (build 14.0-b15, mixed mode)
[root@newton sisma-acquirer]# cat /etc/fedora-release
Fedora release 10 (Cambridge)

EDIT: Solution This solves my problem, I don't know exactly why:

echo 0 > /proc/sys/vm/overcommit_memory

Up-votes for who is able to explain :)

Additional informations, top output:

top - 13:35:38 up 40 min,  2 users,  load average: 0.43, 0.19, 0.12
Tasks: 129 total,   1 running, 128 sleeping,   0 stopped,   0 zombie
Cpu(s):  1.5%us,  0.5%sy,  0.0%ni, 94.8%id,  3.2%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1033456k total,   587672k used,   445784k free,    51672k buffers
Swap:  2031608k total,        0k used,  2031608k free,   188108k cached

Additional informations, free output:

[root@newton sisma-acquirer]# free
             total       used       free     shared    buffers     cached
Mem:       1033456     588548     444908          0      51704     188292
-/+ buffers/cache:     348552     684904
Swap:      2031608          0    2031608

Answer 1

This is the solution but you have to set:

echo 1 > /proc/sys/vm/overcommit_memory

Answer 2

What's the memory profile of your machine ? e.g. if you run top, how much free memory do you have ?

I suspect UnixProcess performs a fork() and it's simply not getting enough memory from the OS (if memory serves, it'll fork() to duplicate the process and then exec() to run the ls in the new memory process, and it's not getting as far as that)

EDIT: Re. your overcommit solution, it permits overcommitting of system memory, possibly allowing processes to allocate (but not use) more memory than is actually available. So I guess that the fork() duplicates the Java process memory as discussed in the comments below. Of course you don't use the memory since the 'ls' replaces the duplicate Java process.

Answer 3

Runtime.getRuntime().exec allocates the process with the same amount of memory as the main. If you had you heap set to 1GB and try to exec then it will allocate another 1GB for that process to run.

Answer 4

This is solved in Java version 1.6.0_23 and upwards.

See more details at http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=7034935

Answer 5

I came across these links:

http://mail.openjdk.java.net/pipermail/core-libs-dev/2009-May/001689.html

http://www.nabble.com/Review-request-for-5049299-td23667680.html

Seems to be a bug. Usage of a spawn() trick instead of the plain fork()/exec() is advised.

Answer 6

I solved this using JNA: https://github.com/twall/jna

import com.sun.jna.Library;
import com.sun.jna.Native;
import com.sun.jna.Platform;

public class prova {

    private interface CLibrary extends Library {
        CLibrary INSTANCE = (CLibrary) Native.loadLibrary((Platform.isWindows() ? "msvcrt" : "c"), CLibrary.class);
        int system(String cmd);
    }

    private static int exec(String command) {
        return CLibrary.INSTANCE.system(command);
    }

    public static void main(String[] args) {
        exec("ls");
    }
}

Answer 7

If you look into the source of java.lang.Runtime, you'll see exec finally call protected method: execVM, which means it uses Virtual memory. So for Unix-like system, VM depends on amount of swap space + some ratio of physical memory.

Michael's answer did solve your problem but it might (or to say, would eventually) cause the O.S. deadlock in memory allocation issue since 1 tell O.S. less careful of memory allocation & 0 is just guessing & obviously that you are lucky that O.S. guess you can have memory THIS TIME. Next time? Hmm.....

Better approach is that you experiment your case & give a good swap space & give a better ratio of physical memory used & set value to 2 rather than 1 or 0.

Answer 8

As weird as this may sound, one work around is to reduce the amount of memory allocated to the JVM. Since fork() duplicates the process and its memory, if your JVM process does not really need as much memory as is allocated via -Xmx, the memory allocation to git will work.

Of course you can try other solutions mentioned here (like over-committing or upgrading to a JVM that has the fix). You can try reducing the memory if you are desperate for a solution that keeps all software intact with no environment impact. Also keep in mind that reducing -Xmx aggressively can cause OOMs. I'd recommend upgrading the JDK as a long-term stable solution.

Answer 9

overcommit_memory

Controls overcommit of system memory, possibly allowing processes to allocate (but not use) more memory than is actually available.

0 - Heuristic overcommit handling. Obvious overcommits of address space are refused. Used for a typical system. It ensures a seriously wild allocation fails while allowing overcommit to reduce swap usage. root is allowed to allocate slighly more memory in this mode. This is the default.

1 - Always overcommit. Appropriate for some scientific applications.

2 - Don't overcommit. The total address space commit for the system is not permitted to exceed swap plus a configurable percentage (default is 50) of physical RAM. Depending on the percentage you use, in most situations this means a process will not be killed while attempting to use already-allocated memory but will receive errors on memory allocation as appropriate.

Answer 10

You can use the Tanuki wrapper to spawn a process with POSIX spawn instead of fork. http://wrapper.tanukisoftware.com/doc/english/child-exec.html

The WrapperManager.exec() function is an alternative to the Java-Runtime.exec() which has the disadvantage to use the fork() method, which can become on some platforms very memory expensive to create a new process.