Using too much Ram in Java

Question

I'm writing a program in java which has to make use of a large hash-table, the bigger the hash-table can be, the better (It's a chess program :P). Basically as part of my hash table I have an array of "long[]", an array of "short[]", and two arrays of "byte[]". All of them should be the same size. When I set my table size to ten-million however, it crashes and says "java heap out of memory". This makes no sense to me. Here's how I see it:

1 Long + 1 Short + 2 Bytes = 12 bytes
x 10,000,000 = 120,000,000 bytes
/ 1024 = 117187.5 kB
/ 1024 = 114.4 Mb

Now, 114 Mb of RAM doesn't seem like too much to me. In total my CPU has 4Gb of RAM on my mac, and I have an app called FreeMemory which shows how much RAM I have free and it's around 2Gb while running this program. Also, I set the java preferences like -Xmx1024m, so java should be able to use up to a gig of memory. So why won't it let me allocate just 114Mb?

You could use a memory profiler to see what exactly is going on. — NPE, Jan 20 '13 at 20:11
Could you try a lesser size? To exclude that it might be some awesome recursion. — Joop Eggen, Jan 20 '13 at 20:20
Could you come up with a minimal self-contained example that you think uses too much space, and post it together with the exact `java` command line you're using? — NPE, Jan 20 '13 at 20:28
Oh, by the way, if I try it with a size of nine-million, it works fine. — Christian Daley, Jan 20 '13 at 20:33
Post code that can reproduce your problem, profile your memory on your own. — Marko Topolnik, Jan 20 '13 at 20:33
Here's an example: Hash_TABLE_SIZE = 10000000; long[] pos = new long[Hash_TABLE_SIZE]; short[] vals = new short[Hash_TABLE_SIZE]; byte[] depths = new byte[Hash_TABLE_SIZE]; byte[] flags = new byte[Hash_TABLE_SIZE]; — Christian Daley, Jan 20 '13 at 20:34
(I deleted my incorrect reply, as I was mixing up normal primitives and arrays of primitives) — sksamuel, Jan 20 '13 at 20:34
I'm using the bluej virtual machine. I know bluej sucks, but I have to use it for my AP Computer Science class. — Christian Daley, Jan 20 '13 at 20:41
You should edit your question instead of posting code in the comment. — Marko Topolnik, Jan 20 '13 at 20:50
BlueJ is an IDE, not a JVM. What JVM are you running? How did you pass `-Xmx1024m` to the JVM? Are you sure the JVM actually picked up that flag? — Matt Ball, Jan 20 '13 at 20:56
@MattBall It is actually a JVM as well---and a language, which is not 100% Java (lacks features). — Marko Topolnik, Jan 20 '13 at 21:02
I tried this at the command line: java -Xms512m -Xmx1024m and I still can't initialize my array. When I try it with size nine-million then it works and says its only using about 105MB — Christian Daley, Jan 20 '13 at 22:56

score 1 · Answer 1 · answered Jan 20 '13 at 21:19

1

You predicted that it should use 114 MB and if I run this (on a windows box with 4 GB)

public static void main(String... args) {
    long used1 = memoryUsed();
    int Hash_TABLE_SIZE = 10000000;
    long[] pos = new long[Hash_TABLE_SIZE];
    short[] vals = new short[Hash_TABLE_SIZE];
    byte[] depths = new byte[Hash_TABLE_SIZE];
    byte[] flags = new byte[Hash_TABLE_SIZE];
    long used2 = memoryUsed() - used1;
    System.out.printf("%,d MB used%n", used2 / 1024 / 1024);
}

private static long memoryUsed() {
    return Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory();
}

prints

114 MB used

I suspect you are doing something else which is the cause of your problem.

I am using Oracle HotSpot Java 7 update 10

answered Jan 20 '13 at 21:19

Peter Lawrey

525,659
79
751
1,130

...like using BlueJ instead of a real JVM. – Matt Ball Jan 20 '13 at 21:21
@MattBall I didn't think I needed to provide the JVM used, but I suspect you are right. – Peter Lawrey Jan 20 '13 at 21:23
Last half hour I've been busy researching BlueJ and its feature set seems to point towards a JVM built on top of JVM. BlueJ is written in Java, but provides many interactive features that resemble dynamic languages, and which can't be replicated in professional IDE's that use production VMs. – Marko Topolnik Jan 20 '13 at 21:30

score 0 · Answer 2 · answered Jan 20 '13 at 20:54

Has not taken into account that each object is a reference and also use memory, and more "hidden things"... we must also take into account also the alignment... byte is not always a byte ;-)

To see how much memory is really in use, you can use a profiler:

visualvm

If you are using standard HashMap (or similar from JDK), each "long" (boxing/unboxing) really are more than 8bytes), you can use this as a base... (use less memory)

NativeIntHashMap

score 0 · Answer 3 · answered Jan 20 '13 at 21:06

From what I have read about BlueJ, and serious technical information is almost impossible to find, BlueJ VM is quite likely not to support primitive types at all; your arrays are actually of boxed primitives. BlueJ uses a subset of all Java features, with emphasis on object orientation.

If that is the case, plus taking into consideration that performance and efficiency are quite low on BlueJ VM's list of priorities, you may actually be using quite a bit more memory than you think: a whole order of magnitude is quite imaginable.

score -2 · Answer 4 · edited May 23 '17 at 12:12

-2

I believe one way it would be to clean the heap memory after each execution, one link is here:

Java heap space out of memory

edited May 23 '17 at 12:12

Community

1
1

answered Jan 20 '13 at 20:15

Layla

5,234
15
51
66

Using too much Ram in Java

4 Answers4