0

We had a peculiar issue in our production system due to a very silly bug in the client library.

What happened was:

public <T> ResponseEntity<T> executePut(String url, JsonObject payload, Class<T> t) {
       String payloadString = payload != null ? payload.toString() : null;
       HttpEntity<String> entity = new HttpEntity<>(payloadString, getCommonHeaders());
       restTemplate.getMessageConverters()
               .add(0, new StringHttpMessageConverter(Charset.forName("UTF-8")));
       return restTemplate.exchange(url, HttpMethod.PUT, entity, t);
   }

As can be seen in the above method, the client library was taking the autowired(singleton) object and kept on adding the element in the MessageConverters list.

Issue: After a while, the reference to the restTemplate object was giving NPE that too with empty stack trace.

PS: I am pointing out this as probable cause because this is the only code change which happened between two deployments.

Also, should not the large object cause heap out of memory? In our case, that too did not happen and other application flows were working flawlessly.

The-Proton-Resurgence
  • 808
  • 1
  • 13
  • 28
  • https://stackoverflow.com/questions/35574184/jvm-issues-with-a-large-in-memory-object – Ng Sharma Nov 21 '19 at 08:03
  • I think the my issue is subtly different from the question in your link, he clearly had HEAP OUT OF MEMORY, but I got no such exception. – The-Proton-Resurgence Nov 21 '19 at 08:06
  • When you run out of memory the stacktrace may not be shown (since there's not enough memory for it). – Kayaman Nov 21 '19 at 08:22
  • Then what makes you think your issue is heap memory-related? I cannot see any indication for that. Try using the debugger and search for where the NPE is thrown. Please adjust your question if you found out more. – Jochen Reinhardt Nov 21 '19 at 08:37
  • @JochenReinhardt That's what I am saying, I have no reason to believe it was a heap issue because the application seemed to be working fine. My question is trying to understand what could have caused the issue? – The-Proton-Resurgence Nov 21 '19 at 09:08

1 Answers1

0

When an object becomes too big so that it consumes all you heap memory, your VM will stop working correctly. It's unlikely that any part of your program running in this VM will keep working correctly.

When the heap is getting full, the garbage collector tries to free memory. And he will try to do so all the time. Your VM will end up doing nothing else. And you will most likely get OutOfMemoryError: GC Overhead Limit Exceeded. See https://www.baeldung.com/java-gc-overhead-limit-exceeded

This error should not be handled by the code in the program. It is unlikely that someone will return null in case of an out of memory error. I would consider this very bad behaviour.

Regarding your NullPointerException: I don't think that it is not having a stack trace. It might not get logged, though. Have you tried attaching a debugger to the system and set an exception break point for NPE?

All the message converters in your REST template will be called one after another. You keep adding message converters to the beginning of the list with every invocation. This does not seem to make sense to me. It might be causing trouble internally.

Jochen Reinhardt
  • 833
  • 5
  • 14