0

i have a python script (runner.py) which regularly measures memory consumption of a process. to monitor memory consumption I use a helper class called ProcessTimer.py. processtimer is implemented based on the solution given here.

the solution is based on polling. so there is a while-loop. in this while-loop memory consumption is measured and then the thread sleeps for a certain amount of time to ensure regular measurements. in my case sleep is done for 500ms. however, my problem is that the measurements are not done each 500ms. the time inbetween measurements seems arbitrary and I don't know why it's not done each 500ms (according to the time.sleep(.500)).

here you can see the CSV output of my measurements. first column is the time (in sec) and second column is the memory consumption:

time,rss_mem
0.0,12.38
1.1,101.89
2.3,110.74
3.4,110.79
4.5,113.61
5.7,101.6
6.8,102.44
8.0,104.28
9.1,104.48
10.3,108.13
11.4,102.64
12.6,102.83
13.7,102.86
14.9,102.86

as you can see there is always ~1.1sec inbetween each measurement, even though the while-loop (which is used for measuring memory consumption of another process) should be executed each 500ms. why is that?

the relevant code of runner.py can be seen here. The helper class ProcessTimer.py can be seen here.

Community
  • 1
  • 1
beta
  • 5,324
  • 15
  • 57
  • 99
  • i just did a log output of the time before and after the `time.sleep(.500)`. and the difference is exactly 500ms. to the `ptimer.poll()` is actually what takes so long. however, i don't know why it takes so long. any suggestions are appreciated. – beta Apr 07 '15 at 12:26

1 Answers1

2

time.sleep(.5) doesn't make sure your loop executes twice per second; it just sleeps half a second. If the rest of the code takes 1 second to execute, each iteration will take .5 + 1 = 1.5 seconds.

If you want to measure in more exact intervals, you will need to measure the time it takes to call ptimer.poll() and then subtract this from .5 seconds.

But from the data above, it seems that ptimer.poll() takes a substantial amount of time. You may have to optimize the memory measurement code first to be able to measure as fast as you want.

Aaron Digulla
  • 321,842
  • 108
  • 597
  • 820
  • yes, `ptimer.poll()` takes too long right now. i have to figure out how to optimize it. if anyone know how to do it, i am happy to read suggestions. thanks for your reply of course. – beta Apr 07 '15 at 12:33
  • 1
    Please look at the code for `poll()`, come up with some ideas and ask a new, specific question. We try to avoid "big" questions here (big = touches more than one topic). – Aaron Digulla Apr 07 '15 at 12:40
  • 1
    okay. problem was the following: to measure cpu usage i used `pp.cpu_percent(interval=1.0)`. so it waited for 1sec. sorry, that was my bad. usage of `pp.cpu_percent()` is explained here: https://code.google.com/p/psutil/wiki/Documentation – beta Apr 07 '15 at 13:01