0

Recently, we upgraded from RHEL 5 to RHEL 6. We have a web application running on JBoss which consumes more memory in RHEL 6 when compared to RHEL 5.

We are using glibc library version 2.12-1.166. We have set the MALLOC_ARENA_MAX varialbe to 1. This reduced the initial memory burst but PermGen contiues to raise. Heapdumps and gc stats analysis doesn't indicate any memory leaks induced by application.

Has anybody experienced a similar issue? I need to get this issue resolved as this is hurting our production servers. We are now restarting 3 times a week to avoid OOM error.

I conducted a memory leak test on RHEL 6 and its evident that malloc is allocating more memory. I tried with MALLOC_ARENA_MAX=1 and MALLOC_ARENA_TEST=1 as suggested in some posts. But it doesn't help.

We are pretty sure the issue is due to the glibc library mentioned above. There are bugs like the one here that suggests that the issue is with glibc library. But we need more information on how to use the glibc environment variables to good effect in solving the issue

Community
  • 1
  • 1
VinayBS
  • 389
  • 5
  • 19
  • Can you roll back to RHEL 5, and then do leak testing on a non-live server? – halfer Apr 13 '16 at 06:20
  • I've tested on RHEL 5 and analysed heap dump and gc stats. It doesnt indicate any leaks in the application. More so, we clearly see increased memory consumption after upgrading to RHEL 6. – VinayBS Apr 13 '16 at 08:46
  • Sure, but that means there's no leaks in the app in RHEL5. Maybe there are leaks in the app in RHEL6 - have you tested that in a non-live env? – halfer Apr 13 '16 at 10:03
  • Yes. I'm testing in the non live RHEL 6 environment using jemalloc to check if there are any memory leaks. I will update here with the output. In the meantime, I wanted to check if anyone has faced similar issues. – VinayBS Apr 13 '16 at 12:28
  • I conducted a memory leak test on RHEL 6 and its evident that malloc is allocating more memory. I tried with MALLOC_ARENA_MAX=1 and MALLOC_ARENA_TEST=1 as suggested in some posts. But it doesnt help.! I'm still stranded. Any help in resolving this issue is appreciated. – VinayBS Apr 14 '16 at 23:45
  • I won't be able to help, I don't use RHEL or Java - just helping with "rubber duck" debugging here. Can you narrow the issue in RHEL 6 down to one component/library? – halfer Apr 15 '16 at 07:40
  • Yes. We are pretty sure the issue is due to the glibc library version 2.12. There are bugs like the one here: [stackoverflow-link](http://stackoverflow.com/questions/27439088/gradual-increase-in-resident-memory-usage-by-jbossjava-process) that suggests that the issue is with glibc library. But we need more information on how to use the glibc environment variables to good effect in solving the issue. – VinayBS May 04 '16 at 01:05

0 Answers0