Increase celery retry time each retry cycle

Question

I do retries with celery like in the Docs-Example:

@task()
def add(x, y):
    try:
        ...
    except Exception, exc:
        add.retry(exc=exc, countdown=60)  # override the default and
                                          # retry in 1 minute

How can I increase the retry-countdown everytime the retry occurs for this job - e.g. 60 seconds, 2 minutes, 4 minutes and so on until the MaxRetriesExceeded is raised?

I think you should change the selected answer. – gdvalderrama Nov 27 '17 at 12:54 — gdvalderrama, Nov 27 '17 at 12:54

Lamp town guy · Answer 1 · 2015-03-23T11:47:19.950

35

Here is a simple way to create bigger delay each time the task is evaluated. This value is updated by celery itself so you don't need to manage anything yourself.

@task()
def add(x, y):
    try:
        ...
    except Exception as exc:
        raise add.retry(exc=exc, countdown=60 * add.request.retries)

Note: First task is repeated with countdown of 0. Because number of retries is 0 for the first run.

edited Mar 23 '15 at 11:47

answered Apr 11 '14 at 14:22

Lamp town guy

1,489
1
16
23

2

Thanks for mentioning *add.request.retries* – Kaveh Aug 09 '17 at 07:00

Anton Shurashov · Accepted Answer · 2018-02-19T13:46:42.907

10

Since version 4.2 you can use options autoretry_for and retry_backoff for this purposes, for example:

@task(max_retries=10, autoretry_for=(Exception,), retry_backoff=60)
def add(x, y):
    pass

edited Feb 19 '18 at 13:46

answered Feb 19 '18 at 13:04

Anton Shurashov

1,820
1
26
39

This does not seem to work when you call `retry()` function (at least on celery 4.2.2). – Sarang Jan 17 '22 at 13:11

score 7 · Answer 3 · answered Feb 27 '12 at 18:23

7

Keep a variable with your last retry time in it, and multiply it by 2 each time until it exceeds whatever level you want (or, keep a count if you prefer a certain number of times...)

answered Feb 27 '12 at 18:23

Perry

4,363
1
17
20

So there is no elegant/magic way to do this instead of passing a retry-counter to the method on every retry? – Gregor Feb 27 '12 at 18:34
I'm insufficiently familiar with celery, but if you've read the manual and haven't found it, this seems like an easy way to accomplish what you want with only two or three lines of code, which seems easy enough. – Perry Feb 27 '12 at 18:36
@Gregor, I think this is elegant, but maybe we could add an retry_step as well. You could create an issue at http://github.com/ask/issue to request such a feature. – asksol Feb 27 '12 at 20:25
17

Oh, btw, you don't have to pass a parameter as the number of retries is available: `interval, step = 60, 60; add.retry(countdown=interval + (step * add.request.retries)` – asksol Feb 27 '12 at 20:27
Thanks asksol, I knew that this counter must be living somewhere, how else would celery raise the MaxRetriesExceeded exception after #retries. – Gregor Feb 28 '12 at 13:15

Increase celery retry time each retry cycle

3 Answers3

Linked