15

In a bash script I need to wait until CPU usage gets below a threshold.

In other words, I'd need a command wait_until_cpu_low which I would use like this:

# Trigger some background CPU-heavy command
wait_until_cpu_low 40
# Some other commands executed when CPU usage is below 40%

How could I do that?

Edit:

  • target OS is: Red Hat Enterprise Linux Server release 6.5
  • I'm considering the average CPU usage (across all cores)
sdabet
  • 18,360
  • 11
  • 89
  • 158
  • There are many posts that can help to get the current CPU usage. My approach would be to keep checking the usage after certain intervals and if the usage in not below threshold keep checking it and once its below the threshold, just to proceed with the rest of the code. I am not a bash coder. Please make me aware if I missed something important? https://unix.stackexchange.com/questions/152988/how-to-get-cpu-usage-for-every-core-with-a-bash-script-using-standard-nix-tools – ρss Aug 31 '15 at 13:05
  • 2
    You could narrow this question down to a particular environment (eg, Linux or Solaris), which will drive the specific tool to use. There is no standard POSIX way to get the CPU utilization. Further, you should clarify what you mean by CPU: since modern CPU have multiple cores, do you consider these each CPU? Do you want an average of the cores? Do you want average across all CPUs' cores? – bishop Aug 31 '15 at 13:06
  • @bishop Thank you for the clarifications. See my edit. – sdabet Aug 31 '15 at 13:08
  • 1
    `yum install sysstat`, then `mpstat -P ALL 1 | grep all | awk '{print $9}'` should get you in the ballpark of the idle % on all CPU. That said, you could also clarify the question by specifying whether the threshold is percentage "idle" or "used". – bishop Aug 31 '15 at 13:14
  • 2
    Adding a small code sample outlining the context you want to use this in, however contrived, might help to dissuade down voters (just something I've noticed with this type of question) – Clive Aug 31 '15 at 13:16
  • @Clive I've added an example of code sample. Hope the question will be reopened. – sdabet Aug 31 '15 at 13:25
  • I would also like to answer, but unfortunately the question is closed. (Good job!) I would use `top` and `awk` to get the value: `usage=$(top -n1 | awk 'NR==3{print $2}'`. Then use `bc` to compare the value against a threshold: `if bc -l <<< "$usage < $thres" ; then echo "usage below thres"; fi` .. Put that into a loop and make a function out of it. – hek2mgl Aug 31 '15 at 13:30
  • @hek2mgl Shall I re-post it as a new question? (there has been many edits since the original submission) – sdabet Aug 31 '15 at 13:45
  • @fiddler I would wait a bit. It should take at maximum a few hours until it is re-opened (just one additional vote is missing).. – hek2mgl Aug 31 '15 at 13:50
  • 1
    **Question:** what do you mean by 40% CPU usage? On most Unix systems, having more that the one CPU means the maximum CPU usage is `100% * NumCPUs`. So is that 40% meaning "40% of maximum CPU usage" or "40% as the system reports it"? – Mr. Llama Aug 31 '15 at 14:50

4 Answers4

11

A much more efficient version just calls mpstat and awk once each, and keeps them both running until done; no need to explicitly sleep and restart both processes every second (which, on an embedded platform, could add up to measurable overhead):

wait_until_cpu_low() {
    awk -v target="$1" '
    $13 ~ /^[0-9.]+$/ {
      current = 100 - $13
      if(current <= target) { exit(0); }
    }' < <(LC_ALL=C mpstat 1)
}

I'm using $13 here because that's where idle % is for my version of mpstat; substitute appropriately if yours differs.

This has the extra advantage of doing floating point math correctly, rather than needing to round to integers for shell-native math.

spawn
  • 192
  • 3
  • 9
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • Brilliant!! Saved my day. Thanks Charles:) – Rajneesh Gadge Jan 14 '20 at 21:21
  • 1
    This script worked perfectly, until under certain mysterious circumstances involving gitlab-runner, mpstat appears to ignore the Broken Pipe when awk exits, and keeps running. Using strace showed every write is returning EPIPE. Just commenting as a warning in case someone else has this problem. – szmoore Dec 17 '20 at 02:47
  • @szmoore, I'd suggest filing a ticket to get this fixed in `mpstat`; it really _should_ consider that error fatal. That said, if you need a workaround in bash for this, I think it should be possible to build one if you have a version of the shell new enough for process substitutions to store their PID in `$!` (I think that feature was introduced in 4.3, maybe?) -- if you ask a new question on the subject, feel free to @ me in. – Charles Duffy Mar 08 '21 at 15:13
  • @CharlesDuffy, I had a really hard time reproducing the issue outside of a gitlab-runner environment. I suspected gitlab-runner blocked the `SIGPIPE` signal somehow but didn't have time to prove who I should be filing the bug report against. For my use case I rewrote the script in Python using `psutil` instead, so it's unfortunately no longer a `bash` question. – szmoore Mar 09 '21 at 12:15
6
wait_for_cpu_usage()
{
    current=$(mpstat 1 1 | awk '$12 ~ /[0-9.]+/ { print int(100 - $12 + 0.5) }')
    while [[ "$current" -ge "$1" ]]; do
        current=$(mpstat 1 1 | awk '$12 ~ /[0-9.]+/ { print int(100 - $12 + 0.5) }')
        sleep 1
    done
}

Notice it requires sysstat package installed.

4

You might use a function based on the top utility. But note, that doing so is not very reliable because the CPU utilization might - rapidly - change at any time. Meaning that just because the check succeeded, it is not guaranteed that the CPU utilization will stay low as long the following code runs. You have been warned.

The function:

function wait_for_cpu_usage {
    threshold=$1
    while true ; do
        # Get the current CPU usage
        usage=$(top -n1 | awk 'NR==3{print $2}' | tr ',' '.')

        # Compared the current usage against the threshold
        result=$(bc -l <<< "$usage <= $threshold")
        [ $result == "1" ] && break

        # Feel free to sleep less than a second. (with GNU sleep)
        sleep 1
    done
    return 0
}

# Example call
wait_for_cpu_usage 25

Note that I'm using bc -l for the comparison since top prints the CPU utilization as a float value.

hek2mgl
  • 152,036
  • 28
  • 249
  • 266
1

As noted by "Mr. Llama" in a comment above, I've used uptime to write my simple function:

function wait_cpu_low() {
  threshold=$1
  while true; do
    current=$(uptime | awk '{ gsub(/,/, ""); print $10 * 100; }')
    if [ $current -lt $threshold ]; then
      break;
    else
      sleep 5
    fi
  done
}

In awk expression:

  • $10 is to get average CPU usage in last minute
  • $11 is to get average CPU usage in last 5 minutes
  • $12 is to get average CPU usage in last 15 minutes

And here is an usage example:

wait_cpu_low 20

It waits one minute average CPU usage is below 20% of one core of CPU.

Jean-Pierre Matsumoto
  • 1,917
  • 1
  • 18
  • 26
  • I like this script the most. One fix: `print $11` needs to be used instead of `print $10` which always throws zero on Ubuntu 20.04. – dominecf Oct 25 '21 at 09:22
  • Another fix: You may want to put some `sleep 60` as the first command in the function; otherwise the function would end immediately if launched simultaneously with some CPU-intensive task (due to load value being averaged). – dominecf Oct 25 '21 at 09:24