0

I've got a Python loop that should run every minute, do some data processing and sleep until the next minute is up. However the processing takes variable amount of time, sometimes it's close to zero when there is not much to do, sometimes it takes even 10 or 20 seconds.

To compensate for that I measure the time it takes to run the processing like this:

while True:
  time_start = time.time()
  do_something()                        # <-- This takes unknown time
  time_spent = time.time() - time_start
  time.sleep(60 - time_spent)

It kind of works but over a couple of days it still drifts away by a number of seconds. I guess it happens when the computer (small Raspberry Pi) is busy and delays the start of the loop, then it all starts slipping away.

I don't need do_something() executed exactly every minute, so no need for real-time OS or anything like that, but I don't want one delayed start affect all the subsequent ones either.

Is there some kind of scheduler that can run my function at a predefined rate? Or some more clever way to compensate for the occasional delays?

KeepLearning
  • 349
  • 2
  • 3
  • 10
  • 1
    if you really want this kind of concurrency I think you have to use threads. one thread as timer and another as worker. – prhmma Nov 03 '19 at 21:30
  • @phalanx what if the timer thread gets delayed / slips away? I don't think it's any different from the timestamp-compensation method I use. Unless you mean some (hardware-)scheduled timer? – KeepLearning Nov 03 '19 at 21:43
  • 1
    How about `time.sleep(60 - time.time() % 60)` as last line? – Michael Butscher Nov 03 '19 at 21:51
  • Possible duplicate of [In Python, how can I put a thread to sleep until a specific time?](https://stackoverflow.com/questions/2031111/in-python-how-can-i-put-a-thread-to-sleep-until-a-specific-time) – quamrana Nov 03 '19 at 21:54
  • @quamrana Thanks, but that's quite an over-complicated solution for a simple fixed-rate trigger that I'm after. My solution below seems to work in my case. – KeepLearning Nov 03 '19 at 23:52

2 Answers2

1

Playing with the loop a little this seems to work quite well. The trick is to record the start time once before the loop starts, not on every iteration. That way one delayed start won't affect any future ones.

rate_sec = 60
time_start = time.time()

while True:
    print("{}".format(datetime.now()))

    # Emulate processing time 0s to 20s
    time.sleep(random.randint(0, 20))

    # Sleep until the next 'rate_sec' multiple
    delay = rate_sec - (time.time() - time_start) % rate_sec
    time.sleep(delay)
KeepLearning
  • 349
  • 2
  • 3
  • 10
0

Is sleeping a pre-requisite of your project? I mean, you don't need to have your processing blocked if you want to run the task every ~1 minute.

Since you are on a Raspberry Pi, you can (and probably should) use crontab.

This will give you the most flexibility and allow you to don't have the computer sleeping idle.

To do this, open your cron tab with

crontab -e

And add the following entry

* * * * * /usr/bin/env python3 /path/to/script.py
weulerfilho
  • 61
  • 4
  • I was considering that but there is quite a lengthy start up procedure and calibration that takes a couple of minutes on RPi, so this is not an option unfortunately. It's ok to do it during RPi boot when the program starts but not on every iteration. Thanks for the suggestion though :) – KeepLearning Nov 03 '19 at 21:41