Issue with Python returning same timestamp several times

Question

I am working with an API that uses a callback function to send data for up to 2000 times per second. The body of the function gets the data argument (that the API calls the function using), appends a timestamp (using datetime.now().timestamp()) to it, then sends it to a queue, where it will be saved in a file after the acquisition has been finished.

The issue I am facing is that I am getting the same timestamps several times, with different data. Below is an example of some of the saved data:

Data------timestamp

3258 1595943590.058758

3246 1595943590.058758

3248 1595943590.058758

3254 1595943590.058758

3246 1595943590.058758

I tried using time.time() instead and the issue was still there:

2986 1595944140.3182354

2984 1595944140.3182354

2986 1595944140.3182354

2982 1595944140.3182354

2980 1595944140.3182354

2986 1595944140.3182354

Is the issue that the API is sending data so fast that the time isnt updating fast enough? Is there a more accurate way to get time?

#part of a class
def apiFunc(self, data):
        if data:
                d = (data, time.time())
                self.storage.put(d)
                return True
        return False

@superbrain Because I want the exact timestamp for each data point for later synchronization and analysis. — Mohamed Moustafa, Jul 28 '20 at 14:06

superb rain · Accepted Answer · 2020-07-28T14:48:12.817

Maybe combine the precision of time.perf_counter with the meaningfulness of time.time. Here's code and output (left is pure time.time, right is my suggestion):

>>> if 1:
    import time
    delta = time.time() - time.perf_counter()
    timestamps = []
    for _ in range(10):
        pure = time.time()
        mine = time.perf_counter() + delta
        timestamps.append((pure, mine))
    for row in timestamps:
        print(row)

(1595947258.0029619, 1595947258.0029738)
(1595947258.0029619, 1595947258.0029812)
(1595947258.0029619, 1595947258.0029826)
(1595947258.0029619, 1595947258.002984)
(1595947258.0029619, 1595947258.0029855)
(1595947258.0029619, 1595947258.0029874)
(1595947258.0029619, 1595947258.0029886)
(1595947258.0029619, 1595947258.00299)
(1595947258.0029619, 1595947258.0029914)
(1595947258.0029619, 1595947258.002993)

Excerpt from a much longer output where the pure time finally changed (had to try thousands of rows):

...
(1595947517.2481732, 1595947517.2532165)
(1595947517.2481732, 1595947517.2532175)
(1595947517.2481732, 1595947517.2532187)
(1595947517.256172, 1595947517.2532716)
(1595947517.256172, 1595947517.253275)
(1595947517.256172, 1595947517.253278)
...

score 0 · Answer 2 · answered Jul 28 '20 at 14:11

According to this answer, time() on windows only has ~16 milliseconds precision.

The standard time.time() function provides sub-second precision, though that precision varies by platform. For Linux and Mac precision is +- 1 microsecond or 0.001 milliseconds. Python on Windows uses +- 16 milliseconds precision due to clock implementation problems due to process interrupts. The timeit module can provide higher resolution if you're measuring execution time.

Issue with Python returning same timestamp several times

2 Answers2