Python scripts that runs every minute without stoping

Question

I have a Python script that sends a post request to my API every 1 minute:

while True:
    data = requests.post("URL_HERE", json={
        "api_key":"XXXX",
        "concat": 1,
        "messages": "HI"
    })
    time.sleep(60)

Everything works fine, but every 2 hours (more or less) there are 2 records of the same minute. Example:

2017-03-22 11:34:46.977255
2017-03-22 11:37:47.231694
2017-03-22 11:37:47.231694
2017-03-22 11:39:48.849003
2017-03-22 11:40:48.907895
...
2017-03-23 13:59:59.150108
2017-03-23 14:00:00.120431
2017-03-23 14:00:00.942033

I guess this is because the code inside the "while" takes a couple milliseconds to execute, thanks to that, one time every 2-3 hours a minute will have two records.

Anyone knows how can I fix this? I can't use cronjobs.

Maybe an asynchronous task?

And If I want this program to execute forever, It is okey to use a "while", or should I create a Daemon o something similar?

checkout celery for these kind of tasks – pycod333 Mar 23 '17 at 14:32 — pycod333, Mar 23 '17 at 14:32

Nils Werner · Accepted Answer · 2017-03-23T14:36:39.673

4

Instead of waiting for 60 seconds you can wait the remaining number of seconds in the current minute:

time.sleep(60 - datetime.datetime.now().second)

If your job finishes late datetime.datetime.now().second will be 1 or 2 and you will wait 59 or 58 seconds.

edited Mar 23 '17 at 14:36

answered Mar 23 '17 at 14:34

Nils Werner

34,832
7
76
98

You're right! This should work. Do you think doing this with a "while" instead of a daemon is ok? – Marcos Aguayo Mar 23 '17 at 14:36
1

A background process, waiting and doing something every once in a while is a daemon. Depending on how reliable you want the job to be you can look into cronjobs or job queues. Judging from not being able to use cron I assume that your environment is very limited and job queues may not be feasible. But that's a different question altogether :-) – Nils Werner Mar 23 '17 at 14:37

score 2 · Answer 2 · edited May 23 '17 at 12:25

2

Take a look at this post. Using a schedular might do the trick. Using that answer on your code will look somethig like this.

import sched, time, requests
schedul = sched.scheduler(time.time, time.sleep)

def send_post():
    data = requests.post("URL_HERE", json={
        "api_key": "XXXX",
        "concat": 1,
        "messages": "HI"
    })

schedul.enter(60, 1, send_post(), (schedul,))
schedul.run()

edited May 23 '17 at 12:25

Community

1
1

answered Mar 23 '17 at 14:39

Joakim

195
1
9

1

I think you are missing another `schedul.enter(...)` inside `send_post()`. – Nils Werner Mar 23 '17 at 14:41
Forgot to mention, if you need to run this over and over you schedule a new run at the end of the first one. (as Nils points out) – Joakim Mar 23 '17 at 14:41
I think you also need `send_post` (not calling the function) and a scheduler argument) – Holloway Mar 23 '17 at 14:43
@Joakim I like this approach, but Nils Werners seem simpler. What's the advantage of using scheduler? – Marcos Aguayo Mar 23 '17 at 14:44
Well it depends on your needs. Nils solution is compact and good and go for it if it suits your needs. You can think of the schedular as Python's own cron job. If you want to learn more check the doc [link](https://docs.python.org/3.6/library/sched.html) – Joakim Mar 23 '17 at 14:49
Actually, this solution will still drift. `schedul.enter(60, ...)` will simply wait for `60` seconds, no matter the current time. You need to calculate the next interval datetime and call `schedul.enterabs(...)`. – Nils Werner Mar 23 '17 at 14:49

score 1 · Answer 3 · answered Mar 23 '17 at 14:43

1

if you keep track of the time you can ensure you don't duplicate or have time drift.

here is an example

interval = 60
next = time.time() + interval

while True:
    if time.time() >= next :
        data = requests.post("URL_HERE", json={
            "api_key":"XXXX",
            "concat": 1,
            "messages": "HI"
        })
        next = next + interval
    time.sleep(next - time.time())

answered Mar 23 '17 at 14:43

Anton Codes

3,663
1
19
28

1

Your solution may skip one cycle if [`time.sleep` returns early](https://docs.python.org/2/library/time.html#time.sleep). A solution is to remove the `if` entirely. – Nils Werner Mar 23 '17 at 14:46
I don't see how that is possible since the sleep cycle uses the difference in time not the absolute time. Can you give a specific example? – Anton Codes Mar 24 '17 at 16:05

Python scripts that runs every minute without stoping

3 Answers3