3

I'm implementing a feature in my dockerized server where it will ignore some of the requests when cpu and memory utilization is too high.

I already found out how to get memory and cpu time used by the process inside container here, but I also need a way to get cpu limit and memory limit (set by Runtime Options) to calculate the percentage.

Also, I can read the load for the host system from /proc/loadavg, is there a "per-container load" somewhere I can read?

I'm using Java/Kotlin on openjdk 14.

Update: I found out I can get memory limit from /sys/fs/cgroup/memory/memory.limit_in_bytes

Pegasis
  • 1,256
  • 11
  • 15
  • 1
    Modern JVMs already know how to set the heap size based on the cgroup memory limits, so this should reduce to the more portable problem of figuring out the current Java heap utilization. – David Maze Jan 03 '21 at 15:57

2 Answers2

10

This is all controlled by cgroups. So you can check the cgroup limits and utilization:

$ docker run -it --rm --cpus 1.5 --memory 1g busybox /bin/sh
/ # cat /sys/fs/cgroup/memory/memory.usage_in_bytes
3043328
/ # cat /sys/fs/cgroup/memory/memory.limit_in_bytes
1073741824
/ # cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us
150000
/ # cat /sys/fs/cgroup/cpu/cpu.cfs_period_us
100000
/ # cat /sys/fs/cgroup/cpu/cpuacct.usage
97206588

The memory usage and limit are fairly straight forward to monitor. This is more accurate than monitoring the Java heap usage since heap is only one portion of memory used by Java.

The CPU quota over the period gives you the number of fractional CPU's allocated, 150000/100000 = 1.5. You could monitor the cpuacct.usage over time to calculate when it exceeds the limit and is throttling your process.

BMitch
  • 231,797
  • 42
  • 475
  • 450
  • I believe memory.usage_in_bytes includes memory used by disk cache? Is there a way to get memory without disk cache? (What about total_rss in memory.stat?) – Pegasis Jan 04 '21 at 19:52
1

For cgroup2:

/ # cat /sys/fs/cgroup/memory.current
503808

/ # cat /sys/fs/cgroup/memory.max
1073741824

/ # cat /sys/fs/cgroup/cpu.stat
usage_usec 58384
user_usec 23602
system_usec 34782
nr_periods 37
nr_throttled 0
throttled_usec 0

/ # cat /sys/fs/cgroup/cpu.max
150000 100000

see: https://facebookmicrosites.github.io/cgroup2/docs/cpu-controller.html

see2: https://facebookmicrosites.github.io/cgroup2/docs/memory-controller.html

sebi88
  • 108
  • 1
  • 6