3

I am generating a large data structure and write it to hard disk. Afterwards I want to get rid of the object, to reduce the memory consumption. My problem is that after I had forced a garbage collection the amount of used memory is at least as high as it was before garbage collection. I have added a minimal working example what I am doing.

DataStructure data = new DateStructure();
data.generateStructure(pathToData);
Writer.writeData(data);
WeakReference<Object> ref = new WeakReference<Object>(data);
data = null;
while (ref.get() != null) {
    System.gc();
}

The code should force a garbage collection on the data object as it is recommended in thread:
Forcing Garbage Collection in Java?
I know this garbage collection does guarantee the deletion of the data object, but in the past I was more successful by using the garbage collection as described at the link as using simply System.gc().

Maybe someone has an answer whats the best way to get rid of large objects.

Community
  • 1
  • 1
markus
  • 57
  • 1
  • 8
  • 4
    Depends on how soon you need the released memory and if the free space before gc is not enough, I would not force it but simply null the reference and let gc do its job on the next tick. – arynaq Jun 20 '13 at 14:11
  • 2
    "the amount of used memory is at least as high as it was before garbage collection" How did you measure that? Are you sure it is the used memory and not the max allowed memory? – Fildor Jun 20 '13 at 14:15
  • Simply I can see the used amount of memory in the system monitor, where it should decrease from almost full to almost empty. Secondly I use runtime.totalMemory() - runtime.freeMemory() to get the real amount. This shows that the heap increases. – markus Jun 20 '13 at 14:36

2 Answers2

7

It seems that this is premature optimization (or rather an illusion of it). System.gc(); is not guaranteed to force a garbage collection. What you are doing here is busy waiting for some non-guaranteed gc to happen. But if the heap does not get filled up the JVM might not start a garbage collection at all.

I think that you should start thinking about this problem when you stress test your application and you can identify this part as a bottleneck.

So in a nutshell you can't really force a gc and this is intentional. The JVM will know when and how to free up space. I think that if you clear your references and call System.gc(); you can move on without caring about whether it gets cleaned up or not. You may read the Official documentation about how to fine-tune the garbage collector. You should rather be using some GC tuning according to the documentation than asking java to GC from your code.

Just a sidenote: the JVM will expand some of the heap's generations if the need arises. As far as I know there is a configuration option where you can set some percentage when the JVM will contract a generation. Use MinHeapFreeRatio/MaxHeapFreeRatio if you don't want Java to reserve memory which it does not need.

Community
  • 1
  • 1
Adam Arold
  • 29,285
  • 22
  • 112
  • 207
  • To get more in detail. The heap might not be filled up - that is true. Your answer brings me even more in trouble. I am using a quite large swap partition and the data structure uses parts of it. With a SSD this is ok if it is used temporarily, but not for all time this slows the complete machine. That is the reason why I don't just want to wait for garbage collection even if the heap is not full. – markus Jun 20 '13 at 14:54
  • You can use a different GC strategy like the one @Marko Topolnik pointed out. – Adam Arold Jun 20 '13 at 15:02
2

This idiom is broken for a whole range of reasons, here are some:

  1. System.gc() doesn't force anything; it is just a hint to the garbage collector;
  2. there is no guarantee when a weak reference will be cleared. The spec says "Suppose that the garbage collector determines at a certain point in time that an object is weakly reachable". When that happens, it is up to the implementation;
  3. even after the weak reference is cleared, there is no telling when its referent's memory will actually be reclaimed. The only thing you know at that point is that the object has transitioned from "weakly reachable" to "finalizable". It may even be resurrected from the finalizer.

From my experience, just doing System.gc a fixed number of times, for example three, with delays between them (your GC could be ConcurrentMarkSweep) in the range of half-second to second, gives much stabler results than these supposedly "smart" approaches.

A final note: never use System.gc in production code. It is almost impossible to make it bring any value to your project. I use it only for microbenchmarking.

UPDATE

In the comments you provide a key piece of information which is not in your question: you are interested in reducing the total heap size (Runtime#totalMemory) after you are done with your object, and not just the heap occupancy (Runtime#totalMemory-Runtime#freeMemory). This is completely outside of programmatic control and on certain JVM implementations it never happens: once the heap has increased, the memory is never released back to the operating system.

Community
  • 1
  • 1
Marko Topolnik
  • 195,646
  • 29
  • 319
  • 436