2

The memory of a ConcurrentHashMap of java-websocket WebSocketClients as values is growing unceasingly.

I've isolated it to the section of code that tries to reconnect to a whitelist of peers every second. I've tested to see if entries are being properly removed after a connection failure, and they are. I've also looked at everything I can find for "concurrenthashmap memory leak".

I have no idea how to implement this possible solution. Does that solution solve the problem? If so, please provide example code.

I tried to implement this suggestion

ConcurrentHashMap<String, MyClass> m =
  new ConcurrentHashMap<String, MyClass>(8, 0.9f, 1);

and I think that slowed the growth rate somewhat, but I have no idea how to tweak it. Is that generally the correct approach? If so, please provide example code.

I also tried switching to a HashTable as recommended here, but I get ConcurrentModificationExceptions immediately, so I think that's out.

How can the memory of this ConcurrentHashMap implementation be managed when there is rapid insertion and removal of thousands of entries per second? If it can't, is there an alternative?

Pass by reference instead of by copy

I added a HashMap that stores a new WebSocket, and a connection can only be re-attempted if it's not present there as well as the ConcurrentHashMap.

This GREATLY improved the memory management, but it still leaks a little, approximately by 0.1 MB per ~5 seconds for about 5 attempts per second.

Is this a problem with my code, WebSocket's "destructor", or ConcurrentHashMap's management of removed values?

Dead Objects?

The growth rate has again been greatly reduced because I remove() from the ConcurrentHashMap before calling close() on the WebSocketClient and then finally remove()ing from the HashMap.

How & why could a WebSocketClient still linger after being closed and removed from both Maps?

System.gc();

The leak rate has been greatly reduced again!

I call it after remove().

Community
  • 1
  • 1
  • 1
    Several Questions: 1.) Which VJM Runtime are you using? 2.) How are you testing the size of the object? 3.) Are there any 'dead' objects being referenced from the ConcurrentHashMap? Use a heap analysis tool to figure this out http://stackoverflow.com/questions/2064427/recommendations-for-a-heap-analysis-tool-for-java for a Heap analysis tool. Regards Perhaps this can help you identify where the – Tinman Feb 01 '14 at 01:56
  • @Tinman Thank you so much Tinman! 1) Either the latest or first to the latest Java SDK, JRE on linux direct from Oracle. 2) `CHM.size()` consistently reports `0`. 3) How exactly would I know that? (I=noob). I will definitely look into heap analysis! Thank you again so very very much and so very much in advance! –  Feb 01 '14 at 02:20
  • 1
    OK, so when I asked how you were testing size, I meant size of that object in memory. The 'removed' Socked Connects will remain in the heap until the Garbage Collector (GC) removes them. The GC will only do this every now and again. – Tinman Feb 01 '14 at 02:45
  • @Tinman Thank you so very much again Tinman! I still don't know how to do that. I'm using VisualVM, but I don't know exactly what to look for. There're a ton of `char[]`s and `String`s in the heap dump. Manually calling `System.gc()` has slowed the rate, but there's still a rate. I'm now using VisualVM, and when I click "Perform GC", it greatly reduces the amount of memory in use. Does it use a "special" GC? Thank you again so very very much in advance! –  Feb 01 '14 at 02:53
  • If you don't know how to check the memory of an object, then how can you claim "The memory of a ConcurrentHashMap of java-websocket WebSocketClients as values is growing unceasingly."? How exactly are you concluding that there is a memory leak? – Tinman Feb 01 '14 at 03:07
  • @Tinman Thank you Tinman! I'm watching it increase in the Ubuntu System Monitor. I'm about to update my answer with what's necessary. I very much appreciate your help! –  Feb 01 '14 at 03:08
  • 1
    Do you already understand everuthing in this blog? http://javarevisited.blogspot.com.au/2011/04/garbage-collection-in-java.html – Tinman Feb 01 '14 at 03:09
  • @Tinman Thank you again Tinman! I just read it, but I'm not sure specifically what I'm looking for. Are those flags still available, or are they already used by default? Thank you very much in advance! Please note my edit. It should be there in a few moments. –  Feb 01 '14 at 03:13
  • let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/46566/discussion-between-tinman-and-gracchus) – Tinman Feb 01 '14 at 03:37

1 Answers1

2

It is highly unlikely that the Map itself is leaking. What is probably happening, if it's related to the map at all, is that you are failing to remove data from the map when you're done with it. It can't get GC'd as long as the map is pointing to it.

keshlam
  • 7,931
  • 2
  • 19
  • 33
  • Thank you keshlam! I `remove()` and check the `size()` which consistently reports near `0`. Could it be possible that a `WebSocketClient` is still around even though it's been removed from the maps? Thank you so much in advance! –  Feb 01 '14 at 02:26
  • @MichaelT Thank you so very much MichaelT! The heap dump shows a lot of `char[]`s and `String`s. When I use its "Perform GC", the memory in use drops dramatically. Is it using something other than `System.gc()`? Thank you so very very much in advance! –  Feb 01 '14 at 02:55
  • If GC is cleaning it up, it isn't a leak. Remember, GC costs cycles, so JVMs often attempt to defer garbage collection until they've used a considerable amount of memory and need to recover it. – keshlam Feb 01 '14 at 03:58