2

I need to use a HashMap whose keys are of Long datatype and values are some user-objects, defined as:

HashMap <Long,SomeClass> dummy=new HashMap<>();

Initially this dummy hashmap contains about 10 million <key,value> pairs, added using 

  dummy.add(SomeLong,new SomeClass(SomeParameters);

The memory consumed is 7-8GB. After creating this map, 80-90% of entries are to be removed:

for (Iterator<Entry<Long, SomeClass>> it = collisionMap.entrySet().iterator(); it.hasNext();) {
    Map.Entry<Long,SomeClass> entry = it.next();         
    if(SomeCondition) {
       it.remove(); 
    }
 }

The memory being used is still same after removing these entries. I checked with Runtime.getRuntime().totalMemory() and Runtime.getRuntime().freeMemory().

Now the problem is why the memory has not been reclaimed after this remove() operation? I am creating this type of hashmap about 1000-2000 times during the course of program. And it is giving java.lang.OutOfMemoryError: GC overhead limit exceeded error :(

Can anyone help? Thanks

***************** Update/Additional information ****************

The SomeClassis defined as:

Class SomeClass {
   private ArrayList <Integer> list1;
   private ArrayList <Integer> list2;    
   public SomeClass(int l,List <Integer> l2) {
     list2=l2;
     list1=new ArrayList<>();
     list1.add(l);
   }     
   public void addList1(int l) { list1.add(l); } 
   public ArrayList <Integer> getList1() { return list1; } 
   public ArrayList <Integer> getList2() { return list2; }    
}

For some reasons, I had earlier planned to use BitSet datatype instead of ArrayList list1. And if I replace list1' by aBitSetvariable and set thel^thbit instead of addingltolist1' and then create dummy HashMap with the objects of this class, then the results about memory are different. Surprisingly, the memory is reclaimed after remove operation on HashMap. E.g. After removing about 80% of entries in HahsMap memory being used is also reduced by 40% approx.

Is it that memory allocated to an ArrayList is not being reclaimed? :( :(

Kaur
  • 279
  • 1
  • 6
  • 18
  • 2
    When you remove an entry the entry itself is removed, but the hashtable remains allocated to its "peak" size. However, this should be almost immeasurable -- the bulk of the space is in the objects you add, and the entries occupy several times the space of the actual hashtable. You're almost certainly seeing objects you "leaked" by somehow chaining them together or some such. – Hot Licks Sep 01 '14 at 03:39
  • You'll need to learn how to use a heap analyzer and find out what part of your program is retaining references to your value objects. – Kevin Krumwiede Sep 01 '14 at 04:32
  • Might be a dupe of [Error java.lang.OutOfMemoryError: GC overhead limit exceeded](http://stackoverflow.com/questions/1393486/error-java-lang-outofmemoryerror-gc-overhead-limit-exceeded) – markspace Sep 01 '14 at 05:16
  • @HotLicks Thanks for the reply. Didn't know that '...ntry itself is removed, but the hashtable remains allocated to its "peak"... '. I am not using these objects in the values corresponding to the keys to be removed - anywhere else. How can I make sure that if I remove a key then the values (objects)corresponding to it should also be available for garbage collection. (I am using `new` operator as shown in e.g above to add objects and there are no implicit references to it) – Kaur Sep 01 '14 at 07:06
  • Trust me, you're barking up the wrong tree. – Hot Licks Sep 01 '14 at 12:02
  • Think it would be interesting how big your memory footprint is without even adding the items to a collection. Another question is why do you add all elements if they are removed anyway? – eiselems Sep 17 '14 at 12:05

1 Answers1

0

java.lang.OutOfMemoryError: GC overhead limit exceeded

This indicates that you're not necessarily using up too much memory (although the map may consume up to 80MB for its internal nodes array in your situation) but that the GC does not have enough breathing room (max heap limit) or CPU time (GC time limit) available to do its work given your allocation profile.

You most likely have to tune your GC parameters to avoid this issue.

the8472
  • 40,999
  • 5
  • 70
  • 122