Do Something At a Consistent Frequency in Python

Question

I have a python program that reads from a serial port, accesses registers then writes data to the CSV. Id like to perform a writing operation to the csv at a set frequency of 100Hz. Below is sample code related to the timing which attempts to limit writing to the csv. For simplicity it will only print to console rather than write to csv.

import datetime
from datetime import timezone 
import time

FREQ = 5
CYCLETIME = 1/FREQ

def main():
    
    while(1):
        ### Code that will read message bytes from a port

        # Time start for Hz
        start = time.monotonic() 
        delta = time.monotonic() - start
        if delta < CYCLETIME:
            time.sleep(CYCLETIME - delta)

        # in the full program we write to a csv but in this simple program we will just print it
        milliseconds_since_epoch = datetime.datetime.now(timezone.utc)
        print(milliseconds_since_epoch)

if __name__ == "__main__":
    main()

2020-08-24 18:57:15.572637+00:00
2020-08-24 18:57:15.773183+00:00
2020-08-24 18:57:15.973637+00:00
2020-08-24 18:57:16.174117+00:00
2020-08-24 18:57:16.374569+00:00
2020-08-24 18:57:16.575058+00:00
2020-08-24 18:57:16.775581+00:00
2020-08-24 18:57:16.976119+00:00
2020-08-24 18:57:17.176627+00:00
2020-08-24 18:57:17.377103+00:00
2020-08-24 18:57:17.577556+00:00

The output seems consistent for 5Hz but if I change it to 100Hz it seems inconsistent. It sounds like this could be a time accuracy drift associated to time.monotonic. Is this a good approach? Is time.montonic appropriate?

My knowledge with pythons thread library is limited but in the near future, I plan to create 2 child threads for each task. One which constantly reads from serial and another that will write (or in our case print to console every 100Hz).

Edit: I took the solution below and modified it. The original solution seemed to only print once. Here is my new attempt:

import datetime
from datetime import timezone 
import time

FREQ = 5
CYCLETIME = 1/FREQ

def main():
    t0 = time.perf_counter()    # Time ref point in ms
    time_counter = t0           # Will be incremented with CYCLETIME for each iteration

    while 1:
        ### Code that will read message bytes from a port

        now = time.perf_counter()
        elapsed_time = now - time_counter
        if elapsed_time < CYCLETIME:
            target_time =  CYCLETIME - elapsed_time
            time.sleep(target_time)
           
        # In the full program we write to a csv but in this simple program we will just print it
        milliseconds_since_epoch = datetime.datetime.now(timezone.utc)
        print(milliseconds_since_epoch)

        time_counter += CYCLETIME

if __name__ == "__main__":
    main()

Output:

I used matplot lib to determine the frequency to create this output. I take the rolling window difference of the current and previous value and inverse it since frequency=1/(time diff).

Timmy Chan · Accepted Answer · 2020-08-25T07:14:27.437

There are 5 problems in your algorithm:

Using time.sleep() has low accuracy, sometimes with an error around 10-13ms, depending on your system. (ref)
start and delta variables do not track the time spent on print(), as delta is set right after start.
start and delta variables use two different time.monotonic(). You should call the function only once, and pass the value to the other variable to make sure the same time value is being used.
Tick rate of time.monotonic() is 64 per second. Use time.perf_counter() instead. (ref)
You are doing time.sleep() on a delta time, which has low accuracy. If you have two threads running together, the cycles will hardly be synchronized. You need a time tracker which makes it sleep based on elapsed time.

Fix:

The following code make use of time.perf_counter(), and uses time_counter to keep track of the previous loop iteration timestamp.

The next loop time is the previous loop timestamp + cycle time - elapsed time. Thus, we will make sure the time.sleep() sleeps the program until then.

def main():
    t0 = time.perf_counter()  # Time ref point
    time_counter = t0  # Will be incremented with CYCLETIME for each iteration

    while 1:
        ### Code that will read message bytes from a port

        now = time.perf_counter()
        elapsed_time = now - t0
        target_time = time_counter + CYCLETIME
        if elapsed_time < target_time:
            time.sleep(target_time - elapsed_time)

        # In the full program we write to a csv but in this simple program we will just print it
        milliseconds_since_epoch = datetime.datetime.now(timezone.utc)
        print(milliseconds_since_epoch)

        time_counter += CYCLETIME

I'll adapt this into the code and let you know if I have any issues. — Potion, Aug 26 '20 at 18:39
I had to make some modifications since the code you posted only prints once (https://hatebin.com/idcsekwbii). Thoughts? If correct, please modify your answer and I will mark the solution as correct. Since you posted the solution first, Id like to award you the answer as you've steered me into the right direction. — Potion, Aug 27 '20 at 21:49
I added a graph. The timing still seems to drift quite a bit for 100 Hz. — Potion, Aug 28 '20 at 17:39
As I mentioned in the answer, time.sleep() has low accuracy. Read more [here](https://stackoverflow.com/a/1133888/7502914). What I have done for you is assuring the program to run at correct interval. Such that after running the program for 100 seconds, you know that it has looped exactly 10000 times. Even though, each iteration is a little bit off due to your OS not being to do accurate sleeping. — Timmy Chan, Aug 28 '20 at 17:56
And if you really want to do accurate timing, consider not doing sleep, and use the perf_counter to check the time on every iteration of loop. However, this approach would be a lot more performance heavy, as you may expect. — Timmy Chan, Aug 28 '20 at 18:00

Do Something At a Consistent Frequency in Python

1 Answers1

Fix: