Java new byte array of size 2^32

Question

In Java, it won't allow me to use a long for something like this:

long size = 0xFFFFFFFF; //2^32-1

byte [] data = new byte[size];

And an int can only go as high as 0x7FFFFFFF (2^31-1). Is it possible to declare a byte array of this size?

Not necessarily a dupe but see this: http://stackoverflow.com/questions/3038392/do-java-arrays-have-a-maximum-size — John, Nov 16 '12 at 18:46
I was going to say that, you can't have more than 2^31-1 elements in an array. And as we can see, in some cases even you can't even reach that. — biziclop, Nov 16 '12 at 18:53

score 10 · Accepted Answer · answered Nov 16 '12 at 18:53

10

Answer is NO as this is the max possible initialization:

    int size = Integer.MAX_VALUE;
    byte [] data = new byte[size];

answered Nov 16 '12 at 18:53

Yogendra Singh

33,927
6
63
73

1

Actually, even doing this is not guaranteed that the array will work, specially if the JVM is working with low levels of memory – Bruno Vieira Nov 16 '12 at 19:13
@BrunoVieira: Indeed, memory is required. – Yogendra Singh Nov 16 '12 at 19:17
See also [Do Java arrays have a maximum size](http://stackoverflow.com/a/8381338/105863)? – Hollis Waite Jun 25 '15 at 22:53

Óscar López · Answer 2 · 2012-11-16T18:58:34.827

3

Declare the size as an int and try again:

int size = 0x7FFFFFFF; // 0x7FFFFFFF == Integer.MAX_vALUE == 2^32-1

An array can only be declared to have a positive int size, not a long. And notice that the maximum int positive value (and hence, the maximum possible size for an integer array, assuming there's enough memory available) is 0x7FFFFFFF == Integer.MAX_vALUE.

edited Nov 16 '12 at 18:58

answered Nov 16 '12 at 18:45

Óscar López

232,561
37
312
386

a 32 bit int cannot hold a value that big though – Eric B Nov 16 '12 at 18:46

score 3 · Answer 3 · answered Nov 16 '12 at 18:57

3

Integer.MAX_VALUE is the limit for many java concepts. For example String/List can't be longer than that.

Back in 1995 when Java was created, 4MB total memory is the norm.

The same problem exists in most languages. But if we are to design a new language today, we'd certainly use long.

answered Nov 16 '12 at 18:57

irreputable

44,725
9
65
93

1

The Java developers were certainly aware of memory sizes much larger than 4GB. The Sun E10000, which shipped in 1997 and was a planned Sun product from July 1996 on, supported up to 64 GB of memory. It was in development within Cray Research, based on Sun processors and known to Sun, before that. – Patricia Shanahan Nov 16 '12 at 20:27

Bruno Vieira · Answer 4 · 2012-11-16T19:12:09.303

2

The problem in your example is that the array create function expects an int as argument and you were passing a long.

This compiles:

long size = 0xFFFFFFFF; //2^32-1

byte [] data = new byte[(int)size];

But doesn't run. It doesn't run in this particular example of long because the number is negative, and you simply can't create an array with negative number of elements. But still, it's possible to use a long variable to create an array, but it need to be a positive number and need to be cast to int, like the following:

long size = 0x00FFFFFF; //

byte [] data = new byte[(int)size];

Which will compile and work.

By the way, changing from size to Integer.MAX_VALUE won't accomplish nothing if this exceeds your JVM memory limits.

edited Nov 16 '12 at 19:12

answered Nov 16 '12 at 18:44

Bruno Vieira

3,884
1
23
35

I don't see how that could possibly work, isn't that int negative, or INT_MAX or something? it can't be actually 2^32-1 after the cast. – Eric B Nov 16 '12 at 18:46
The memory management is (most of the time) a problem of the JVM. – Bruno Vieira Nov 16 '12 at 18:47
1

It compiles but gives `java.lang.NegativeArraySizeException` at second line as `(int)size` -> `-1`. – Yogendra Singh Nov 16 '12 at 18:50
You see, that's because you are passing a negative int as parameter. And as such, it isn't possible to create an array with negative elements. It compiles because cast is available only on runtime and that's when it says it is a NegativeArraySizeException – Bruno Vieira Nov 16 '12 at 18:56
With this long which is > Integer.MAX_VALUE it won't work. But it's possible to create and use a long variable and then create an array casting it to int. – Bruno Vieira Nov 16 '12 at 19:00
I edited the post to correct my previous mistakes. I had focused only on the creating of an array using `long` and not on the size of the same. The new answer add more relevant information – Bruno Vieira Nov 16 '12 at 19:14
Better. :) Seemed the main problem was more the value than the operand size, so. – cHao Nov 16 '12 at 20:06

Patricia Shanahan · Answer 5 · 2014-04-28T13:27:50.493

2

I agree that you cannot have a byte[] with more than Integer.MAX_VALUE elements. However, you can have an array of arrays.

I suggest writing a class that has:

A constructor that takes a long size and builds a byte[][] big enough to contain that many elements.
Get and set methods that use a long index
A size() method that returns a long.

Of course, this is only practical with a 64 bit JVM and a big enough memory.

I felt even back in the 1990's that the choice of int rather than long for the array index and length types was short-sighted. Sun was selling many machines with tens of gigabytes of memory, and there has been a consistent tendency both for maximum array sizes to approach memory sizes, and for memory sizes to grow.

edited Apr 28 '14 at 13:27

answered Nov 16 '12 at 19:00

Patricia Shanahan

25,849
4
38
75

java was meant for running applets on consumer PCs:( – irreputable Nov 16 '12 at 19:02
I struggle to find any practical use for a 4GB array. If you need one, that's probably a design error. – biziclop Nov 16 '12 at 19:18
@biziclop Here's the first example I thought of - representing a genome with one byte per base-pair. Even the human genome, at 3 billion base pairs, would be too large for a single Java array. Some plants have much larger genomes. – Patricia Shanahan Nov 16 '12 at 20:18
I should also add that back in the 1970's I heard similar objections to arrays bigger than 64K elements. There has been a continuous history of parallel increases in memory sizes and maximum array sizes. – Patricia Shanahan Nov 16 '12 at 20:33
@PatriciaShanahan And why would an object-oriented language be good for that kind of pure data crunching? Horses for courses. While the maximum RAM available for a VM doesn't surpass the terabyte limit, a single object larger than 4G makes little sense. When it will start to make sense, changing array indexing to long will be a relatively painless affair. – biziclop Nov 16 '12 at 21:52
I have been told that a single object bigger than 64 KB does not make sense, even if some computers have a few megabytes of memory. What is different today? Each array size goes through a phase of being physically supportable and needed for some applications, but discounted as being too big to make sense by some programmers, on its way from being impossible to being normal. – Patricia Shanahan Nov 16 '12 at 23:58

score 1 · Answer 6 · answered Nov 16 '12 at 18:53

1

This is not possible. Array size supports only int values.

You could check the JDK source code for arrays. Only int values are supported.

answered Nov 16 '12 at 18:53

messivanio

2,263
18
24

1

You don't need the source code, the JLS makes it quite clear that array indexes can only be `int` or other types that can be promoted to `int`. – biziclop Nov 16 '12 at 18:55

Java new byte array of size 2^32

6 Answers6