1

We have two micro services. one with django and mysql for user actions and another one with bottle and mongodb for geo calculations.

From the background we need to update/look at both mysql and mongodb. How can we do this in celery ? tasks can be execute from Django app or bottle app.

Thanks.

Karesh A
  • 1,731
  • 3
  • 22
  • 47
  • yes it can be done with celery. You'll need a broker to handle the celery tasks and a worker. Que two separate jobs one for each dB and you should be good to go – cph Apr 12 '17 at 03:40
  • what is the best way to do update the mongodb with celery ? I didnt find any good soltions with mongodb and celery ? ok. but how can we keep consistence in two tasks updates. I need to update mysql first and update mongodb. – Karesh A Apr 12 '17 at 05:29

1 Answers1

0

Use 2 celery taskes. You'll need a broker.

For Mysql that is well documented. Use https://docs.mongodb.com/ecosystem/drivers/python/ to connect to the mogodb and update the record.

Link the two tasks together.
http://docs.celeryproject.org/en/master/userguide/canvas.html#chains

Set up celery to do exponential retries. Will help avoid the thundering herd problem. Increase celery retry time each retry cycle

Community
  • 1
  • 1
cph
  • 458
  • 2
  • 6
  • 24