3

For a process, I have set a soft limit value of 335544320 and hard limit value of 1610612736 for the resource RLIMIT_AS. Even after setting this value, the address space of the process goes up to maximum 178MB. But I am able to see the value of the soft and Hard limits in /proc/process_number/limits correctly set to the above said values.

I wanted to know whether RLIMIT_AS is working in my OS and would also like to know how I can test for the RLIMIT_AS feature.

CentOS 5.5(64 bit) is the operating system that I am using.

Some please help me with regard to this. Thank you!

NorthCat
  • 9,643
  • 16
  • 47
  • 50
Rajath
  • 31
  • 1
  • 2
  • 1
    Err... 335544320 = 320MB. Your process is quite a bit lower than that. What exactly is the problem? – thkala Mar 05 '11 at 07:53
  • I would like to know why the process is not raising up to the value that I have set(320MB) instead of limiting itself to 178MB. I am in need to increase the size of the process in 64 bit systems. I want a process to grow to a larger size when RAM is available. – Rajath Mar 05 '11 at 07:57
  • @Rajath: It's up to the process how much memory to allocate, as long as it doesn't go over your limit. If you want it to have larger buffers or something, that will depend on exactly what you are running. – Jeremiah Willcock Mar 05 '11 at 08:03
  • could some one please let me know how OS uses this soft and hard limit values. I meant whether the process is allocated 320MB in my case while the process gets started or the OS allocates as and when memory is needed by the process. – Rajath Mar 05 '11 at 08:18
  • @Rajath: The soft limit is the actual resource limit and the hard limit is the maximum of what an unprivileged process can set its soft limit to. But they are both *limits*. That's it. It's up to your process to grow/shrink/whatever *within those limits*. – thkala Mar 05 '11 at 08:25
  • @thkala: Okay then why isn't my process within the soft(320MB) and hard limits(1.5GB). Currently the process size is 170MB. Do you mean to say that RLIMIT_AS is not working. – Rajath Mar 05 '11 at 08:30
  • @Rajath: *both* the soft and the hard limit are *upper limits*. The hard limit is simply a limit for the value of the soft one. – thkala Mar 05 '11 at 08:34

1 Answers1

6

All setrlimit() limits are upper limits. A process is allowed to use as much resources as it needs to, as long as it stays under the soft limits. From the setrlimit() manual page:

The soft limit is the value that the kernel enforces for the corresponding resource. The hard limit acts as a ceiling for the soft limit: an unprivileged process may only set its soft limit to a value in the range from 0 up to the hard limit, and (irreversibly) lower its hard limit. A privileged process (under Linux: one with the CAP_SYS_RESOURCE capability) may make arbitrary changes to either limit value.

Practically this means that the hard limit is an upper limit for both the soft limit and itself. The kernel only enforces the soft limits during the operation of a process - the hard limits are checked only when a process tries to change the resource limits.

In your case, you specifiy an upper limit of 320MB for the address space and your process uses about 180MB of those - well within its resource limits. If you want your process to grow, you need to do it in its code.

BTW, resource limits are intended to protect the system - not to tune the behaviour of individual processes. If a process runs into one of those limits, it's often doubtful that it will be able to recover, no matter how good your fault handling is.

If you want to tune the memory usage of your process by e.g. allocating more buffers for increased performance you should do one or both of the following:

  • ask the user for an appropriate value. This is in my opinion the one thing that should always be possible. The user (or a system administrator) should always be able to control such things, overriding any and all guesswork from your application.

  • check how much memory is available and try to guess a good amount to allocate.

As a sidenote, you can (and should) deal with 32-bit vs 64-bit at compile-time. Runtime checks for something like this are prone to failure and waste CPU cycles. Keep in mind, however, that the CPU "bitness" does not have any direct relation with the available memory:

  • 32-bit systems do indeed impose a limit (usually in the 1-3 GB range) on the memory that a process can use. That does not mean that this much memory is actually available.

  • 64-bit systems, being relatively newer, usually have more physical memory. That does not mean that a specific system actually has it or that your process should use it. For example, many people have built 64-bit home file servers with 1GB of RAM to keep the cost down. And I know quite a few people that would be annoyed if a random process forced their DBMS to swap just because it only thinks of itself.

thkala
  • 84,049
  • 23
  • 157
  • 201