Challenges getting simple Huey example started with Crontab

Question

I'm attempting to implement the suggested organization of code from the Huey docs into an existing app, and also following the simple example. The goal is to build a crontab that will run the task every day at 3:00 am.

I powered up two terminal tabs, the first one with the consumer running the script from the example:

PYTHONPATH=.:$PYTHONPATH
export WORKER_CLASS=${1:-thread}
huey_consumer.py main.huey --workers=4 -k $WORKER_CLASS -C -S

Then, in the other tab, I run the main.py script:

python main.py

config.py

from huey import SqliteHuey

huey = SqliteHuey(filename='/tmp/huey.db')

tasks.py

from config import huey

# Note that this time is 1 minute before whenever I'm debugging.
# I'm using the 3 am example as what we're aiming for in our final product.
@huey.periodic_task(crontab(minute='0', hour='3'))
def run_this_function():
    system = New_Class() # Class instantiation since there's a bunch that happens when a new class is set up 
    system.run_method # Runs a bunch of methods from the class in one location

main.py

from config import huey
from tasks import run_this_function

def main:
    run_this_function()

if __name__ == "__main__":
    main()

The task runs immediately, and as I'm brand new to Huey, not sure what I might be missing to make it work on a schedule. I've tried so many crontab combinations, not sure if the challenge is there, or in how I call the run_this_function in the main method. Any help is appreciated!

This is arguably too broad. https://stackoverflow.com/questions/22743548/cronjob-not-running has a large number of troubleshooting tips for `cron` jobs. (I notice you fail to export `PYTHONPATH` but there may be other things to check.) Simply running `main.py` will call `run_this_function()` by way of the standard `__name__` logic you have at the end. — tripleee, Aug 12 '20 at 06:56
Thanks for the Cronjob info. I’m looking specifically for the Huey Python library implementation here. With the Huey task wrapper, it should call the function but set that job up to run later, and I’d like to keep everything in the library if I can. — Rich Everts, Aug 12 '20 at 23:20
Thanks for clarifying. I'm not familiar with Huey and I guess I didn't understand your question properly, but I think I have a better idea now of what you are trying to ask. Are you sure the `tasks` job won't *also* run it when the time comes? You probably don't want `main` to call the function at all, just `import tasks` or otherwise evaluate the code in it. — tripleee, Aug 13 '20 at 04:33
I submitted an issue about the wacky `export`: https://github.com/coleifer/huey/issues/545 — tripleee, Aug 13 '20 at 04:40
So; if you start running `huey_consumer.py main.huey --workers=4 -k thread -C -S` slightly before 3 AM and then run `tasks.py` does it not trigger the function at 3 AM? — tripleee, Aug 13 '20 at 04:43
The `tasks.py` is not called anywhere. If I do execute `tasks.py` via console, nothing happens with the running Huey task manager. If run `main.py`, it will immediately call the `periodic_task(crontab)` function. It does NOT run it again at 3am. In the example, the author uses ```result = run_method() \n print('Result:') \n print(result.get(True))``` But that still executes the function right away, and returns nothing at the scheduled time. If it helps at all, the `periodic_task(crontab)` calls a class method that has a bunch of print statements. That's how I determine if it runs. — Rich Everts, Aug 13 '20 at 15:28
Have you been running `huey_consumer.py` at 3 AM and observed that the scheduled job does not trigger when the time comes? Please [edit] the question itself to clarify this. If it doesn't look roughly like "I started `huey_consumer.py` at 02:55 and then ran `import task` in a separate Python process. I waited until 03:00 and a little bit more but nothing happened" then you need to do that first. (Probably pick a more convenient time in the near future so you don't have to stay up just for this.) — tripleee, Aug 13 '20 at 15:34
Sorry, I always debug as you mentioned above. Far easier. Added the edits above! — Rich Everts, Aug 14 '20 at 15:35

score 1 · Accepted Answer · answered Sep 10 '20 at 23:14

So first of all, you don't actually want to call run_this_function() yourself, so there's no need to run your main.py script in another tab. You only need to have the huey consumer running as you want it to be responsible for executing the task at requested times.

Your consumer needs to be able to discover the task, which you can do by importing it to that main file as you're doing and then starting the huey instance through that file (which you are also doing). A simple test could be putting a print statement in the same file as where you're defining the periodic task. Once you run the consumer, you should see your print statement. If not, your task won't be picked up by the consumer either.

Secondly, I would suggest not using the crontab function. I have not been able to get it working in the past and instead you're better of writing your own function to configure the 3 am. Huey calls the supplied function regularly with the current timestamp. So as a reusable example you could do something like:

def run_at_my_preferred_hour(hour_to_run):
    last_day_it_ran_for = None

    def check_if_task_needs_to_run_for_this_time(current_timestamp):
        if current_timestamp.hour == hour_to_run and current_timestamp.day != last_day_it_ran_for:
            # Save that you already ran the task for this day
            last_day_it_ran_for = current_timestamp.day
            return True
        return False

    return check_if_task_needs_to_run_for_this_time

    
@huey.periodic_task(run_at_my_preferred_hour(3))
def run_this_function():
   ...

Challenges getting simple Huey example started with Crontab

config.py

tasks.py

main.py

1 Answers1