1

What exactly is it measuring?

I have an Debian VM in Azure with 16 vCPUs. I am using it to run tensorflow. The metric "Percentage CPU" on Azure Portal shows 33.5% average. My concern is that I might not fully utilize all the 16 vCPUs.

What really puzzles me is that the top command shows a dominating Python process of 600% CPU. Why is this number not consistent with Azure's 33.5%?

At one point, I was suspecting the number of tensflow threads is not enough. However, when I increased the tensorflow threads from 5 to 15, Azure "Percentage CPU" remained unchanged at 33.5%. top -H did show the increased number of threads though.

UPDATE:

See the accepted answer below. In the top screen, type 1, it shows all the CPUs are used:

enter image description here

So Azure's "Percentage CPU" metric is %CPU you see with top command. Average meaning the average %CPU of all the CPUs.

Shui shengbao
  • 18,746
  • 3
  • 27
  • 45
neurite
  • 2,798
  • 20
  • 32
  • Does this imply only 6 out of the 16 vCPUs are used? The math is that 1 vCPU is counted by `top` as 100% CPU. So 600% shown by `top` means 6 vCPUs are used. In the mean time, 6/16 = 37.5% which is close to Azure's 33.5% average. – neurite Dec 14 '17 at 23:32
  • In `top` screen, enter `1`, you will see single CPU usage. – Shui shengbao Dec 15 '17 at 01:32

2 Answers2

2

Per Azure support team, Azure basic metric "Percentage CPU" shows how much of the physical node the Guest OS (running your program) is actually using. So 33% means it is actually using around 5 vCPUs fully. Note that the extended metric "CPU Percentage Guest OS" shows what the Guest OS thinks is being used when the Guest OS in running.

jatsl
  • 21
  • 2
1

Does this imply only 6 out of the 16 vCPUs are used?The math is that 1 vCPU is counted by top as 100% CPU. So 600% shown by top means 6 vCPUs are used.

No, I don't think so. 16 vCPUs are all used.

You could check this answer.

%CPU -- CPU Usage : The percentage of your CPU that is being used by the process. By default, top displays this as a percentage of a single CPU. On multi-core systems, you can have percentages that are greater than 100%. For example, if 3 cores are at 60% use, top will show a CPU use of 180%. See [here][2] for more information. You can toggle this behavior by hitting Shifti while top is running to show the overall percentage of available CPUs in use.

You could execute top, then enter 1. You will see single CPU usage.

top - 01:32:05 up 9 min,  1 user,  load average: 0.15, 0.38, 0.33
Tasks: 110 total,   1 running, 109 sleeping,   0 stopped,   0 zombie
%Cpu0  :  1.0 us,  0.0 sy,  0.0 ni, 87.8 id, 11.2 wa,  0.0 hi,  0.0 si,  0.0 st
Shui shengbao
  • 18,746
  • 3
  • 27
  • 45
  • You mean all the 16 vCPU are used such that 33.5% * 16 = 550%? If so, why using only 33% of a single vCPU? The entire process takes 8 hours to finish. I would like to see it using 100% CPU instead and finish in half of the time. – neurite Dec 15 '17 at 05:44
  • @neurite Yes, you could type `1` to check single CPU usage. Single CPU usage depends on OS CPU allocation principle. – Shui shengbao Dec 15 '17 at 05:48
  • @neurite `I would like to see it using 100% CPU instead and finish in half of the time.` It also depends on your algorithm. Based on my knowledge, using multiple process could reduce running time. The right algorithm can improve CPU and memory usage and reduce runtime. – Shui shengbao Dec 15 '17 at 05:54
  • @neurite This link http://careers.directi.com/display/tu/Understanding+CPU+Utilization+and+Optimization I think will help you understanding CPU utilization. – Shui shengbao Dec 15 '17 at 05:58
  • @neurite Hi, could you see single CPU usage? – Shui shengbao Dec 15 '17 at 06:15
  • Fine, if you have any question, you could post here. – Shui shengbao Dec 15 '17 at 06:23
  • Yeah, you right about the CPU usage. All the 16 vCPUs are used. Now the question is why it tops at 33%. But thanks a bunch for the clarification. Answer accepted. – neurite Dec 15 '17 at 06:25