0

My project might be an overcommitted system, and I have to improve the reliability by specifying an appropriate container mem limit, by which the total mem of the node should not be divided. But I'm confused with the following statements in the Kubernetes v1.1 doc Resource of Qos:

Incompressible Resource Guarantees

  • if they exceed their memory request, they could be killed (if some other container needs memory)

  • Containers will be killed if they use more memory than their limit.

and the command docker stats shows a "LIMIT" for each container:

enter image description here

I think it means that containers will not use mems more than the "LIMIT" since I've met sometimes the MEM% stays at 100% for a while, so how and when the containers are killed?

Update

I think OOM Killer is enabled with the default value 0.

> cat /proc/sys/vm/oom_kill_allocating_task
0
Matthias M
  • 12,906
  • 17
  • 87
  • 116
Yang
  • 759
  • 2
  • 9
  • 30

1 Answers1

1

Cgroup memory limit feature is used, so I recommend to read cgroup doc:

Tasks that attempt to consume more memory than they are allowed are immediately killed by the OOM killer.

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/sec-memory.html

Jan Garaj
  • 25,598
  • 3
  • 38
  • 59
  • Thanks! Actually, my containers are used to computation, the max mem will be `16G`, so the container will use more the `8.59G` limit. So how to explain the `100% LIMIT` in the `docker stats`? It stays for a while and not any containers are killed. – Yang Feb 29 '16 at 09:41
  • Is OOM killer enabled? – Jan Garaj Feb 29 '16 at 09:45
  • Isn't it enabled by default? I think it might be not the problem, but the fact that containers will **not** consume more memory than they are allowed, according to the "LIMIT" column. – Yang Feb 29 '16 at 14:08
  • the OOM killer is enabled, please check the `Update` in the question. – Yang Mar 01 '16 at 02:08