0

I have a Celery task that runs the main function of external Python scripts. However, some of the scripts may take a very very long time to execute and I'd like to set a time limit after which the tasks are aborted if they're not finished.

I've tried all the ways I've found on other similar questions like this one but none of the things I tried worked.

My task is the following :

@app.task(name="execute_script", bind=True, base=CallbackTask)
def execute_script(self, script_id, variables, name, start_time):
    self.update_state(state='EXECUTING')
    script = Script.objects.get(pk=script_id)
    mod = Script.get_script(script) # Getting the script's module
    try:
        mod.main(variables, name)
    except Exception as e:
        raise e
    return name

And my Celery settings :

import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'main.settings')

app = Celery('main', backend='django-db')

app.config_from_object('django.conf:settings', namespace='CELERY')

app.autodiscover_tasks()

@app.task(bind=True)
def debug_task(self):
    print('Request: {0!r}'.format(self.request))

Is there a way to set a time limit on a Celery task after which it's aborted if not finished ?

Balizok
  • 904
  • 2
  • 5
  • 19
  • Does this answer your question? [Setting Time Limit on specific task with celery](https://stackoverflow.com/questions/11672179/setting-time-limit-on-specific-task-with-celery) – DejanLekic Jul 08 '22 at 09:47
  • I've tried this method but that didn't work. I may have done it wrong but I don't know where I messed up – Balizok Jul 08 '22 at 09:50
  • Can you describe what exactly "didn't work" when you tried that? What is the behavior you are observing and what is the behavior you want? Do you want the task to just be dropped? Or do you want another worker to retry the same task later? Can you please show your settings? – sytech Jul 11 '22 at 19:53
  • I want my task to be dropped, like if a task is never ending I want it to be dropped in order not to run forever, without being re ran or anything. And when I tried the suggested solution, there was absolutely no change, I tried putting a time-limit of 10 seconds on my task, but it was still running way over that time. I've updated the question with my Celery settings. – Balizok Jul 12 '22 at 07:07

1 Answers1

0

Remove the base=CallbackTask, then try the time-limits.

Maybe it's the parent task that makes this method not work.

954
  • 66
  • 3