Exception Java heap space

Question

This is my code:

public static void runSGD(double[] R, double[][] theta, double convergenceTol)
{
    List<Integer> allEdges = new ArrayList<Integer>(2*E);    
    for (int i = 0; i < 2*E; i++) 
        allEdges.add(i);
    Collections.shuffle(allEdges, new Random(shuffleSeed));
    double oldRes = calcObj(R, theta, allEdges), newRes = 0.0;
    long numEdges = 0;
    for (int _e = 0; _e < 2*E*tp; _e++) {
        int e = allEdges.get(_e);
        numEdges += weights.get(e);
    }
    if (verbose)
        System.out.printf("[Info] Number of edges in training, including multiplicity = %d\n", numEdges);
    int[][] edgeTable = new int[4][1<<30];
    long part = 0; int cur = 0;
    for (long i = 0; i < numEdges; i++) {
        if (i+1 > part) {
            part += weights.get(allEdges.get(cur));
            cur++;
        }
        int row = (int) (i >>> 30);
        int col = (int) (i & ((1 << 30) -1));
        edgeTable[row][col] = allEdges.get(cur-1);
    }
}

The error is Exception in thread "main" java.lang.OutOfMemoryError: Java heap space when run this code:

 int[][] edgeTable = new int[4][1<<30];

I have try -Xmx1g,-Xmx3g, but didn't work ,how to fix it?

Stephen C · Accepted Answer · 2018-11-11T07:32:21.743

2

You are allocating 4 int[] arrays of 2^30 ints. That is 2^34 bytes or 16 gigabytes. Clearly that won't fit into a 1 or 3 gigabyte heap. Indeed, a typical laptop or PC won't have enough RAM for this ...

There is a secondary problem about whether the heap spaces are large enough to hold a 2^32 byte object, but it should be possible to address that if you can make the heap large enough.

In fact the JVM supports arrays of just under 2^31 elements; see Do Java arrays have a maximum size?, so the array size per se is not the problem here.

edited Nov 11 '18 at 07:32

answered Nov 10 '18 at 09:58

Stephen C

698,415
94
811
1,216

1 int 4 = 2^2 bytes 2^30 int 2^32 bytes how come 2^34 bytes could you explain? – Soner from The Ottoman Empire Nov 10 '18 at 10:02
1

4 `int[]` arrays each containing 2^30 `int` values. Each `int` is 4 bytes. 4 x 4 x 2^30 == 16 x 2^30 == 16 gigabytes. – Stephen C Nov 10 '18 at 10:04

Max7cd · Answer 2 · 2018-11-10T10:12:51.283

0

Arrays use ints to address single array elements. The maximum int-value is 2^31-1. You create 4 Arrays of the size 2^30 which means you have 2^32 elements in your array. Java does simple not support arrays of that size.

You can fix it by making 4 distinct arrays.

Also allocating 3GB with -Xmx3g wont help, since your array alone will need 16GB of RAM.

edited Nov 10 '18 at 10:12

answered Nov 10 '18 at 09:49

Max7cd

94
6

score 0 · Answer 3 · answered Nov 10 '18 at 12:30

0

As others mentioned, you are trying to allocate huge arrays. You can try by allocating chunks of small array or try collections.

answered Nov 10 '18 at 12:30

mani deepak

401
1
4
15

Exception Java heap space

3 Answers3