0

After a lot of effort I can't seem to overcome the problem of getting a

GC overhead limit exceeded

error in my Java program.

It occurs inside a large method that contains large string manipulation, many lists of objects and accesses to DB. I have tried the following:

  1. after the use of each ArrayList, I have added: list=new ArrayList<>(); list=null;
  2. for the strings, instead of having e.g. 50 appends (str+="....") I try to have one append with the total text
  3. after each DB access I close the statements and the resultSets.

This method is called from main like this:

for(int i=0; i<L; i++) {
    cns = new Console(i);

    cns.processData();//this is the method

    cns=null;
}

When this loop gets executed 1 or 2 times, everything is ok. For L>=3 it's almost certain that I will get the garbage collector error.

Shouldn't the fact that I have a cns=null after each execution of the method, force the GC and free everything from the previous execution?

Should I also delete all private attributes of the object before setting it to null? Maybe putting a Thread.sleep() could force the GC after each loop?

Paul T.
  • 4,703
  • 11
  • 25
  • 29
hragber
  • 1
  • 1
  • 2
  • Probably you third run occupies too much memory. I would suggest to split your large method into smaller ones. Calling GC manually and Thread.spleep is a wrong approach. – Admit Oct 28 '17 at 21:55
  • Doing ``list=new ArrayList<>(); list=null;`` just doubles the number of lists that have to be garbage collected. You're creating a new ArrayList and then immediately setting it to null. – NomadMaker Feb 14 '21 at 06:11
  • If you have a lot of appends for a String, use a StringBuilder. – NomadMaker Feb 14 '21 at 06:12
  • I've only made one problem where calling garbage collection manually was useful, but that was due to speed rather than such a lack of space. Without manually calling the gc, there would be odd pauses in the gui. System.gc() in the event loop solved the problem. – NomadMaker Feb 14 '21 at 06:16

2 Answers2

0

There's actually no reason to set cns to null at the end of each loop. You're setting it to a new Console() at the beginning of the loop anyway - if anything could be freed by setting it to null, it's also going to be freed by setting it to a new object.

You can try a System.gc(); call to suggest the system do a garbage collection, but I don't know if that would help you or make it worse. The system IS already attempting garbage collection - if it wasn't, you wouldn't get this error.

You don't show us exactly how you're building your Strings, but keep in mind that += isn't the only culprit. If you have something like String s = "Hello " + "to the" + " world";, that's just as bad as putting that on three lines and using +=. If it's an issue, StringBuilder may be your friend.

You can read the answers at Error java.lang.OutOfMemoryError: GC overhead limit exceeded for some other suggestions on how to avoid this error. It seems that for some people it's triggered when you're almost, but not quite, out of memory. So increasing the amount of memory available to Java may (or may not) help.

D M
  • 1,410
  • 1
  • 8
  • 12
  • Thanks for your answer. The StringBuilder tip helped me. Also, on the DB side. Apart from closing the statement and ResultSet after each statement, is it a good practice to also close the Connection once in a while? Would that save memory? – hragber Oct 29 '17 at 15:48
  • It might save a small amount of memory, but unless you're doing significant processing or waiting for user input in between database calls, it may not be worth it. If you keep closing connections and making new ones, that just means more garbage collection. – D M Oct 29 '17 at 16:48
  • Using ``String s = "Hello " + "to the" + " world";`` is fine. These are all constants and the compiler will optimize it. It's only when there are variables (perhaps from the database) there problems occur. – NomadMaker Feb 14 '21 at 06:14
  • In fact ... in that example the compiler does the concatenation at compile time. – Stephen C Jul 15 '22 at 11:25
  • @StephenC I guess I could have made an example with some variable in the middle instead... – D M Jul 15 '22 at 18:44
  • Well yea ... you could. But using a `StringBuilder` to replace a one-line concatenation will not help either. In earlier Java releases, the `javac` compiler did that for you. In current releases, `javac` emits `invokedynamic` code for the entire expression that the JIT compiler can optimizer better than you can do yourself; see https://dzone.com/articles/jdk-9jep-280-string-concatenations-will-never-be-t – Stephen C Aug 18 '22 at 01:50
  • @StephenC Interesting. "for simple string concatenations..., explicit use of StringBuilder and StringBuffer will actually preclude the ability for the compiler to make use of the JEP 280-introduced feature discussed in this post." So that part of my answer may be out of date and even counterproductive now, even if it helped OP at the time. – D M Aug 18 '22 at 19:05
0

Basically, an "GC overhead limit exceeded" is a symptom of having too much reachable data. The heap is filling up with things that cannot be garbage collected ... 'cos they are not garbage! The JVM is running the GC again and again in an attempt to make space. Eventually, it decides that too much time is being spent garbage collecting, and it gives up. This is usually the correct thing to do.

The following ideas from your question (and the other answer) are NOT solutions.

  1. Forcing the GC to run by calling System.gc() won't help. The GC is already running too often.

  2. Assigning null to cns won't help. It immediately gets something else assigned to it. Besides, there is no evidence that the Console object is occupying much memory.

    (Note that the constructor for the java.io.Console class is not public, so your example is not meaningful as written. Perhaps you are actually calling System.getConsole()? Or perhaps this is a different Console class?)

  3. Clearing private attributes of an object before setting it to null is unlikely to make any difference. If an object is not reachable, then the values of its attributes are irrelevant. The GC won't even look at them.

  4. Calling Thread.sleep() will make no difference. The GC runs when it thinks it needs to.

The real problem is ... something that we can't determine from the evidence that you have provided. Why is there so much reachable data?

In general terms, the two most likely explanations are as follows:

  1. Your application (or some library you are) is accumulating more and more objects in some data structure that is surviving beyond a single iteration of the for loop. In short, you have a memory leak.

    To solve this, you need to find the storage leak; see How to find a Java Memory Leak. (The leak could be related to database connections, statements or resultsets not being closed, but I doubt it. The GC should find and close these resources if they have become unreachable.)

  2. Your application simply needs more memory. For example, if a single call to processData needs more memory than is available, you will get an OOME no matter what you try to get the GC to do. It cannot delete reachable objects, and it obviously cannot find enough garbage to fast enough.

    To solve this, first see if there are ways modify the program so that it needs less (reachable) memory, Here are a couple of ideas:

    • If you are building a huge string to represent the output before writing it to an OutputStream, Writer or similar. You would save memory if you wrote directly to the output sink.

    • In some cases, consider using StringBuilder rather than String concatenation when assembling large strings. Particularly when the concatenations are looped.

      However, note that 1) in Java 8 and earlier javac already emits StringBuilder sequences for you for concatenation expressions, and 2) in Java 9+, javac emits invokedynamic code that is better than using StringBuilder; see

If that doesn't help, increase the JVM's heap size or split the problem into smaller problems. Some problems just need lots of memory to solve.

Stephen C
  • 698,415
  • 94
  • 811
  • 1,216