Is this a good practice to mitigate OutOfMemoryError?

Question

Consider a java process, in that the main thread reads Json objects from a source and assigns a task with the read Json objects to a FixedThreadPoolExecutor with 5 worker threads.

Here the problem is, the source from which the Jsons are read is much faster than the completion of the tasks by the worker thread. So, as the Jsons accumulate in memory waiting for the worker threads, OutOfMemoryException is thrown. Here the number of worker threads cannot be increased.

So will the following be an efficient solution to this OutOfMemory problem?

In main thread check the percent of available memory and sleep for some time if the memory usage is more than 80%

main(){
    for(;;){
       //read the data and assign to executor
      long total = Runtime.getRuntime().maxMemory(),free = Runtime.getRuntime().freeMemory()
      float usedPersent = (total-free)*100/total
      if(usedPercent > 80)
         Thread.sleep(2000);
     }
}

Will there a good practice?

Marko Topolnik · Answer 1 · 2013-12-05T10:48:32.023

You should solve your OOME problem with proper design instead of hacks. Your JSON objects should be pushed into a bounded blocking queue, which will automatically take care of applying "backpressure" against the producer.

An even cleaner option would be to redesign so that each of your submitted tasks is given a portion of the loaded JSON to process, so it does not pull anything on its own. This way you don't even need to implement the queue: you rely on the queue internal to your Executor Service, which you only need to configure appropriately. For example, use an explicit ThreadPoolExecutor constructor, passing it the CallerRunsPolicy as Rejection Policy.

Also note that loading too much JSON in advance will slow down your system overall because the excess objects will be promoted to the Old Generation, increasing the frequency of Major Garbage Collections.

score 1 · Answer 2 · edited May 23 '17 at 12:13

1

No - this is a bad idea.

You should create your thread pool with a maximum pool size and a limited length queue size and handle rejections properly.

There's a post here that looks promising. It professes to be a BoundedExecutor.

edited May 23 '17 at 12:13

Community

1
1

answered Dec 05 '13 at 09:34

OldCurmudgeon

64,482
16
119
213

Can't the client be made to block instead of being rejected? – Marko Topolnik Dec 05 '13 at 09:35
@MarkoTopolnik - I hope so - still investigating. – OldCurmudgeon Dec 05 '13 at 09:36
1

It appears that the `CallerRunsPolicy` is a good approach. "This provides a simple feedback control mechanism that will slow down the rate that new tasks are submitted." – Marko Topolnik Dec 05 '13 at 09:37

score 1 · Answer 3 · answered Dec 05 '13 at 09:35

1

No it's a bad idea. As far as I know the behaviour of a JVM is simply not specified after an OutOfMEmoryException occured (sorry, I don't have the links right now). You might not even see this exception in some cases. So it is essential to monitor JVM heap usage, e.g. in server environments.

answered Dec 05 '13 at 09:35

spa

5,059
1
35
59

Yes, after both an OOME and SOE the behavior is generally unpredictable. In practice, however, we rarely shut down our systems at the first sight of these errors because they are fully recoverable from in at least 90% of the cases. – Marko Topolnik Dec 05 '13 at 09:40

score 0 · Answer 4 · answered Dec 05 '13 at 09:35

0

A better solution would probably be a queue that blocks when it's full. That way, you keep the backlog manageable without the funkiness that is thread synchronization via Thread.sleep.

answered Dec 05 '13 at 09:35

cHao

84,970
20
145
172

score 0 · Answer 5 · answered Dec 05 '13 at 09:41

I am not sure about that idea. Where does you main thread retrieve its work from? Is it keeping clients slept and waiting as well during those 2 seconds?

I think you have 2 ways to attack this problem:

Do you think you have any way to improve the efficiency of your data processing? Maybe if you can make your worker threads to perform better they can relieve the load on your system faster than they do currently
Or, if you are sure that you can not get a better performance, maybe yours is a genuine resources problem and should ask your operator or whoever that manages the production system for a different JVM setup, with a different heap size. Have a look at

http://www.jafaloo.com/java-xmx-memory-settings/

If you provide them with a sound estimation of the amount of memory you need, they should be reasonable (yes, I have treated with tough operators in my life :P)

Is this a good practice to mitigate OutOfMemoryError?

5 Answers5