0

I'm trying to create a very large matrix, but it does not fit in the java heap memory (it consumes up to 8gb of memory using the -xmx command). It is a matrix of around 3 million x 3 million, but it only has 0's and 1's, and is of type byte.

byte [][] uniones = new byte [nube.length][nube.length];

is there any way to get a light matrix?

myradio
  • 1,703
  • 1
  • 15
  • 25
Yair Camborda Morocho
  • 89
  • 6
  • 6
    it has been asked many times, just search for sparse matrices in java. What do you want to do with it? https://stackoverflow.com/questions/989378/data-structure-for-storing-sparse-matrices https://stackoverflow.com/questions/390181/sparse-matrices-arrays-in-java – DPM Aug 28 '19 at 20:56
  • im making a triangulation of around 3million of points, and I relate them using 0 & 1. – Yair Camborda Morocho Aug 28 '19 at 20:59
  • 3
    [`BitSet`](https://docs.oracle.com/javase/8/docs/api/java/util/BitSet.html). Or, specifically `BitSet[]` for a matrix. – Elliott Frisch Aug 28 '19 at 21:00
  • 1. Why is it 8gb and not 9mb (3mb x 3mb)? Should fit in memory if you increase the initial heap size accordingly. 2. Why not use bits if it's binary? – Alon Segal Aug 28 '19 at 21:05
  • 1
    @AlonSegal Actually, 3million x 3million = 9trillion (9tb), not 9 billion (9gb). Even packed into bits instead of byte, you'd use more than 1 Tb of memory. I believe [DPM is correct](https://stackoverflow.com/questions/57699907/how-to-create-a-very-large-matrix-with-0-1-without-java-memory-exception#comment101843811_57699907), for something that huge, you'd need a sparse matrix. – Andreas Aug 28 '19 at 21:09
  • i set memory heap in 10240m (10gb) and doesnt work, how can i use the bits? – Yair Camborda Morocho Aug 28 '19 at 21:10
  • Logically a boolean value would work, but it does not necessarily use less memory than a `byte`. From [Primitive Data Types](https://docs.oracle.com/javase/tutorial/java/nutsandbolts/datatypes.html) about `boolean`: _This data type represents one bit of information, but its "size" isn't something that's precisely defined._ It depends on the JVM implementation. – Kaan Aug 28 '19 at 21:12
  • 2
    @YairCambordaMorocho Read my last comment. 3 million x 3 million is 9 **trillion** bits, i.e. more than **1 Tb** of memory. See links by DPM for how to do sparse matrices instead. – Andreas Aug 28 '19 at 21:12
  • It sounds like you want to compute the values as needed. Even computing the whole matrix will take a while. –  Aug 28 '19 at 21:14
  • To sum up: (0) Do not make a naked array the interface of your matrix; make it an object that can address (get, set) specific elements by method parameters. (1) Use bits, not bytes, for 1s and 0s. (2) Use sparse mattrices to represent large swaths of 0s and 1s; even simple RLE could help reduce the space requirements of such a matrix dramatically. – 9000 Aug 28 '19 at 21:21

0 Answers0