I'm trying to set up a workflow with celery that creates groups of tasks that run in parallel, executing the groups in a specific order. Using Celery v3.1.20 and redis as backend. Some tasks in each group are OK to fail, the other steps are not really dependent on having every one of the tasks succeed in order to complete the workflow.
Something like this:
@celery_app.task(name="tasks.chordfinisher")
def chordfinisher(*args, **kwargs):
return "ok"
@celery_app.task(name="tasks.test_task")
def test_task(idx):
t = int(idx[5:])
if not t % 3:
raise ValueError
time.sleep(random.random() * 1.5)
logging.info('[%s] begin' % idx)
def test_chain():
g1 = group(test_task.si("foo::" + str(i)) for i in xrange(10))
g2 = group(test_task.si("bar::" + str(i)) for i in xrange(10))
step1 = chord(g1, chordfinisher.si())
step2 = chord(g2, chordfinisher.si())
# need to wait for all of step1's tasks to COMPLETE
# before starting step2
workflow = (step1 | step2)
workflow()
Its not possible to sync on groups (also this question) so thats why I made them chords with a pointless callback. The problem is, when one of the tasks fails in the first group, the subsequent groups in the chain (step2
above) do not execute. Is there some way to tell celery to carry on regardless?