How much CPU should a Python time.sleep(n) call use

Question

I have a programme running on an old laptop which is constantly monitoring a Dropbox folder for new files being added. When it's running the Python process uses close to 50% of the CPU on a dual-core machine, and about 12% on an 8-core machine which suggests it's using close to 100% of one core). This is giving off a lot of heat.

The relevant bit of code is:

while True:
    files = dict ([(f, None) for f in os.listdir(path_to_watch)])
    if len(files) > 0:
        print "You have %s new file/s!" % len(files)
        time.sleep(20)

In the case that there is no new file, surely most of the time should be spent in the time.sleep() waiting which I wouldn't have thought would be CPU-intensive - and the answers here seem to say it shouldn't be.

So two questions:

1) Since time.sleep() shouldn't be so CPU-intensive, what is going on here?

2) Is there another way of monitoring a folder for changes which would run cooler?

Your `time.sleep` is indented too far, put it in the main `while True` block, this makes sure it will always run. — Wessie, Aug 01 '13 at 13:30

Foon · Accepted Answer · 2013-08-01T13:30:30.513

3

1) Your sleep only gets called when there are new files.

This should be much better:

while True:
    files = dict ([(f, None) for f in os.listdir(path_to_watch)])
    if len(files) > 0:
        print "You have %s new file/s!" % len(files)
    time.sleep(20)

2) Yes, especially if using linux. Gamin would be something I'd recommend looking into.

Example:

import gamin
import time
mydir = /path/to/watch
def callback(path, event):
    global mydir
    try:
        if event == gamin.GAMCreated:
            print "New file detected: %s" % (path)
            fullname = mydir + "/" + path
            print "Goint to read",fullname
            data = open(fullname).read()
            print "Going to upload",fullname
            rez = upload_file(data,path)
            print "Response from uploading was",rez
    except Exception,e: #Not good practice
        print e
        import pdb
        pdb.set_trace()


mon = gamin.WatchMonitor()
mon.watch_directory(mydir, callback)
time.sleep(1)
while True:
    ret = mon.handle_one_event()
mon.stop_watch(mydir)
del mon

edited Aug 01 '13 at 13:30

answered Aug 01 '13 at 13:25

Foon

6,148
11
40
42

Wow, head-slap moment on the indenting problem! And I'll take a look at `gamin`, though I'm currently on Windows. – Jamie Bull Aug 01 '13 at 13:32
I'm 99% sure there's a Windows equivalent through the Windows API, although you might need to use IronPython. On the other hand, I realized after I posted, the real benefit to gamin etc. is that you get essentially instant updates; if your main concern is just making the CPU not burn up, your polling pattern with sleep makes far more sense to just keep it simple. (And we've all been there with the head-slap, although theoretically, python should be easier to spot... I still remember a C program where I had a hanging else that looked fine with the indents I had) – Foon Aug 01 '13 at 13:39
The problem is the part I posted was actually correct. It was giving the new file/s a chance to get synchronised on Dropbox. The real time.sleep() was further down and wasn't (quite) so easy to spot, but was still incorrectly indented. – Jamie Bull Aug 01 '13 at 13:41

score 2 · Answer 2 · answered Aug 01 '13 at 13:31

2

There is also a cross platform API to monitor file system changes: Watchdog

answered Aug 01 '13 at 13:31

Rahul K

665
10
25

How much CPU should a Python time.sleep(n) call use

2 Answers2