Reducing Big USED RuntimeMemory after a process

Question

I have a process that collect data object from database (hibernate+oracle) and save them to another database the amount of data object near 10,000.

This takes 1000+ Runtime memory. And this memory doesn't reduced after finished the process.

The memory status and the Thread status printed after finished below...

        Runtime runtime = Runtime.getRuntime();
        total = runtime.totalMemory() / MB;
        free = runtime.freeMemory() / MB;
        used = total - free;
        max = runtime.maxMemory();

        System.out.println("Memory Status >>> [ Used: " + used + " MB, Free: " + free + " MB, Total: "+total+" MB ]");


    Set<Thread> threadSet = Thread.getAllStackTraces().keySet();
    for (Thread t : threadSet) {
        System.out.println(t.getState() +" ("+ t.isAlive() + ") : " +t.getName()+" || " +Thread.currentThread().getName() );
    }

The output

Memory Status >>> [ Used: 1262 MB, Free: 719 MB, Total: 1981 MB ]

RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} WAITING (true) : http-bio-8123-exec-1 RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} WAITING (true) : http-bio-8123-exec-16 RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} WAITING (true) : http-bio-8123-exec-11 WAITING (true) : Finalizer RUNNABLE (true) : main WAITING (true) : Java2D Disposer RUNNABLE (true) : http-bio-8123-Acceptor-0 RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} WAITING (true) : http-bio-8123-exec-4 RUNNABLE (true) : http-bio-8123-exec-2 RUNNABLE (true) : ajp-bio-8009-Acceptor-0 RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} WAITING (true) : Reference Handler WAITING (true) : http-bio-8123-exec-17 TIMED_WAITING (true) : ajp-bio-8009-AsyncTimeout RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} TIMED_WAITING (true) : ContainerBackgroundProcessor[StandardEngine[Catalina]] RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} WAITING (true) : File Reaper WAITING (true) : http-bio-8123-exec-10 RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} WAITING (true) : http-bio-8123-exec-20 RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} TIMED_WAITING (true) : GC Daemon WAITING (true) : http-bio-8123-exec-12 TIMED_WAITING (true) : http-bio-8123-AsyncTimeout RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} RUNNABLE (true) : Signal Dispatcher WAITING (true) : http-bio-8123-exec-19 RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211} TIMED_WAITING (true) : Timer-0 RUNNABLE (true) : Memcached IO over {MemcachedConnection to /192.168.39.205:11211}

How can I reduce USED memory, And how can I find out, who use this big memory ?

score 1 · Answer 1 · answered Oct 25 '17 at 06:47

The java memory model takes as much memory as it can get. The -Xmx flag can set a maximum. The GC only runs if this maximum is close to be reached. You can trigger it through System.gc(), but still it is not forced to free all the memory.

To sum up, there is no need that the memory is released after a huge operation until the next one arrives. The GC is really optimized to do just the best to keep the memory clean.

If you think you have a memory leak (the memory cannot be freed by the GC) run your expensive operation more than once and see if the memory gets released. The JDK has a tool named jvisualvm which can show you the memory state and even trigger a GC to see how the memory gets released.

Here is a nice article which explains a bit how the idea is: https://www.cubrid.org/blog/understanding-java-garbage-collection

If there is memory leakage issue, then how can I detect this. — Mr. Mak, Oct 25 '17 at 11:51

score 1 · Answer 2 · answered Oct 26 '17 at 06:32

The best way is to enable verbose GC logs, and then track the timings at which full GC is triggered which basically covers the GC for young, survivor and tenured generation.

So still after the full GC, if you see that the memory is not freed up, then the best tool will be to use eclipse MAT and track which objects are occupying memory.

Lets say if you find a Object X of class X which holds most memory, then you can analyse the object or the class. Eclipse MAT will also give you option to see what data your object is holding.

So I would suggest you not to go through by this runtime memory data that you are printing. But enable the verbose GC logs. Analyse them, and still if you find any issue then analyze through profiling tools like JvisualVM, eclipse MAT, Java Mission Controller etc

Reducing Big USED RuntimeMemory after a process

2 Answers2