0

Currently we are seeing our processes taking too long with mmap call. Once the process reaches to roughly ~2.8 GB, the mmap call takes upto 100 seconds and its being killed by heart beat mechanism built in the process. Would like to know anyone has seen this issue or know why would mmap take more than 100 seconds when asked for memory. In all the cases the stack trace looks the same but memory is allocated in different parts of the code.

Host and compiler info:

  • Host memory: 70 gb OS: redhat 6.3 compiler: gcc 4.4.6 process memory limit(32 bit): 4 gb No Swap configured

And when this happens the host still has 50GB of memory left.

Stack Trace:

#0  0x55575430 in __kernel_vsyscall ()
#1  0x560f9dd8 in mmap () from /lib/libc.so.6
#2  0x5608f2db in _int_malloc () from /lib/libc.so.6
#3  0x5608fb7e in malloc () from /lib/libc.so.6
#4  0x55fb509a in operator new(unsigned int) () from /usr/lib/libstdc++.so.6
#5  0x55f91ed6 in std::basic_string<char, std::char_traits<char>, std::allocator<char> >::_Rep::_S_create(unsigned int, unsigned int, std::allocator<char> const&) ()
   from /usr/lib/libstdc++.so.6
user2461517
  • 52
  • 4
  • 2
    2.8 GB could be 32-bit unsigned integer overflow related. – Baum mit Augen Apr 28 '14 at 16:25
  • You should compile your program (and setup your system) as 64 bits. – Basile Starynkevitch Apr 28 '14 at 16:50
  • The upper limit was tested with a test program, we could reach upto ~4GB . – user2461517 Apr 28 '14 at 17:01
  • 1
    @user2461517 I'd be surprised if that were true, since Linux reserves 1GB of the total maximum address space of 4GB for the kernel. Unless you've modified your kernel, 3GB of user address space is what's available to 32-bit applications. If you need more than that, it's time to begin the journey to 64-bit... – twalberg Apr 28 '14 at 20:21
  • @twalberg, we are using 64 bit OS so the process should have access to ~4GB. – user2461517 Apr 29 '14 at 15:03
  • @user2461517 The info posted above does not support that assertion - the "Host and compiler info" bit specifically states a 32-bit 4GB memory limit, and your stack trace shows 32-bit addresses, not 64-bit ones. You may have a 64-bit OS, but it appears that particular program is compiled as a 32-bit program. – twalberg Apr 29 '14 at 15:05
  • @twalberg thx for the reply. Wrote a test program to allocate 1MB and memsets to '0' in a loop. When it reaches ~4GB it Aborts. `code`#0 0x00110425 in __kernel_vsyscall () #1 0x00a28af1 in raise () from /lib/libc.so.6 #2 0x00a2a3ca in abort () from /lib/libc.so.6 #3 0x002ff327 in __gnu_cxx::__verbose_terminate_handler() ------------------------------------------------------------- TOP Output: ---------- PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 23820 admim 20 0 4093m 4.0g 896 S 0.0 12.7 0:04.95 a.out – user2461517 Apr 29 '14 at 15:52
  • @user2461517 That is to be expected of a 32-bit program... Not sure what else you expected... Note that the values under VIRT/RES/SHR do not indicate that you have allocated 4GB, but that the total process size is 4GB, including mapped libraries, kernel, etc... – twalberg Apr 29 '14 at 16:00
  • [link](http://stackoverflow.com/questions/5079519/memory-limit-to-a-32-bit-process-running-on-a-64-bit-linux-os) suggests that process can have upto 4GB. – user2461517 Apr 29 '14 at 16:52

0 Answers0