How to create a very large 2D Matrix in Java?

Question

I have tried the following way:

int[][] Matrix=new int[Integer.MAX_VALUE-5][Integer.MAX_VALUE-5];

But I get this exception: Exception in thread "main" java.lang.OutOfMemoryError: Java heap space.

What i mean is i want to store a matrix of nxn size where n is around 500000, but my Netbeans shows out of memory exception for just 10000, how would i increase the memory

Do you have a requirement to handle such a big data chunk using Java? Or does it have to be "A" HL language? — ha9u63a7, Nov 06 '14 at 08:23
Is there are problem to be solved or do you just fiddle with java? — Hannes, Nov 06 '14 at 08:24
This would require about 16 Petabyte = around 16,400 Exabyte = 16,700,000 Terabyte = 17,200,000,000 Gigabyte. This number cannot even be displayed in (signed) 64-bit integers! — king_nak, Nov 06 '14 at 08:25
In response to your edit: http://stackoverflow.com/questions/15460779/how-to-increase-the-java-heap-size-in-netbeans Although the array you're trying to create is still unfeasible. Why do you need to do this? — Forager, Nov 06 '14 at 08:33

score 4 · Answer 1 · answered Nov 06 '14 at 08:24

4

There is no way that array will fit in memory.

Integer.MAX_VALUE is 2^31.

Two dimensions means storing 2^62 integers at 4 bytes each.

answered Nov 06 '14 at 08:24

Thilo

257,207
101
511
656

score 2 · Accepted Answer · answered Nov 06 '14 at 08:49

2

You should have a look at a sparse matrix, which allows you to process huge matrices containing only relatively few values.

Examples of a sparse matrix format:

CSR (Compressed sparse row) -- stores non-zero elements in contiguous array with another two arrays containing column indices and pointers to row beginnings
COO (Coordinate list) -- stores array of tuples (row, column, value)

answered Nov 06 '14 at 08:49

Erbureth

3,378
22
39

This is what I was looking for. Thank you so much. Long live Erbureth :) – Adeetya Ravisankar Nov 06 '14 at 15:46

score 1 · Answer 3 · answered Nov 06 '14 at 08:40

For 10000 x 10000 set your heap size to 512MB: java -Xmx512m. For 500000 x 500000 set your heap size to about 1TB: java -Xmx1000000m. You might want to wait a few years for computers with that amount of memory to become available for the general public...

How to create a very large 2D Matrix in Java?

3 Answers3