2

I am working on a big application deployed in Glassfish 4. I have one job which is downloading files and storing them in the database as byte[]. There are lot of small files (like 5 Mb). I got an out of memory error so I tried to start the job again with smaller number of files and started monitoring it with jVisualVM and noticed this: enter image description here

Between 1:30 and 1:40 it is the normal application run, we can see that everything looks as expected. However when I start my job after 1:40 the used heap is often garbage collected so there are a lot of spikes... However, the Heap size is getting lower and lower. I guess this was the cause of the out of memory error. But I have no idea what that means - what does it mean whe the heap size is being lowered?

trincot
  • 317,000
  • 35
  • 244
  • 286
Wolf
  • 871
  • 1
  • 7
  • 21
  • What was the message of the `OutOfMemoryError` exception? – M A Jul 02 '15 at 21:41
  • Depending on the number of small files, I can foresee those hogging up a ton of memory on the server before they're successfully written and flushed to the database. If you have a POJO representing that file, consider marking the byte array in which they come in as [`transient`](http://stackoverflow.com/questions/910374/why-does-java-have-transient-variables). – Makoto Jul 02 '15 at 21:46
  • "Heap size is getting lower and lower" - AFAIK This is allocated heap, OS/Kernel maintains this allocation depending on demand for memory from other process (If it is NOT consumed by Java process), lowering this value at runtime might not be related to OutOfMemoryError. – kosa Jul 02 '15 at 21:49
  • java.lang.OutOfMemoryError: Java heap space (but I am sorry I can't put the stack trace here, company policy) .... @Makoto well, it is flushing them because that insert is being done in new transaction and I can access them immediately after download – Wolf Jul 02 '15 at 21:50
  • @Nambari but this only happens when I run the job, in the picture you can see that I started it again and it again started lowering the allocated heap... what can it mean? Because when the allocated heap is lowered enough, there is no more memory for new objects – Wolf Jul 02 '15 at 21:52
  • @Wolf: My guess is when you run job, other process on the machine might be demanding more memory, due to which OS might shared "available"/"non-consumed" heap with those processes. I remember reading similar theory, couldn't find any references at this point. – kosa Jul 02 '15 at 21:54
  • 1
    @Nambari hmm interesting... still it doesn't look like that to me (but I am very inexperienced in this) but after it finishes, the memory is stil lowered and does not go back to the previous size itself.. the bump in the middle of 1:50 to 2:00 when it returns back, that is when I ran one method in the application, it's our own method called reloadCache but I think it does more (trying to find that out exactly now) ... but it doesn't look like it's soemthing from the outside to me – Wolf Jul 02 '15 at 22:08
  • @Wolf: OutOfMemoryError happens in case where JVM can't allocate memory for the state of the object, which means "heap used" should go based on your description, which is not happening based on the chart, so I am pretty confident that my theory makes sense. To prove this you could try one more thing, try to monitor overall machine resource consumption while job triggered, which in my guess would be higher than normal cases. – kosa Jul 03 '15 at 03:26

0 Answers0