4

I'm considering a move to Elastic Beanstalk (on account of the pricing). The blockage is that I have no idea how to setup Celery on a python app (Django, in my case) deployed to the service. Has anyone managed to setup celery on Elastic Beanstalk? If so, please let me know how you managed to do it and what tools you used.

Pat
  • 1,198
  • 3
  • 12
  • 22

1 Answers1

7

Use the SQS service. Read this: Celery with Amazon SQS

and this:

http://docs.celeryproject.org/en/latest/getting-started/brokers/sqs.html

Community
  • 1
  • 1
GivP
  • 2,634
  • 6
  • 32
  • 34
  • 6
    did u manage to have it working? my problem is: how do i run celery on the EB? – EsseTi Nov 11 '12 at 11:39
  • You're not really supposed to run other services like Celery on EB. It's meant to run your main app only. You should probably run your Celery service on a separate EC2 instance. That's what I do, anyway. – GivP Nov 15 '12 at 05:40
  • I am not sure if Beanstalk has changed much since this was initially posted. Looking at their docs looks like you should be able to run daemon processes with beanstalk. You can create a custom AMI with celery daemon configured. I was able to get it to work with a custom AMI using SQS as the broker. – Vijay Selvaraj Jul 19 '13 at 17:07
  • There is also a way of doing this without creating a custom AMI, see my anwer to a similar question: http://stackoverflow.com/questions/14761468/ – yellowcap Mar 20 '14 at 13:01
  • If a new instance boots due to auto scaling in EB then that new instance will also have celery worker running, in that case is there any chance of multiple execution of same message in SQS ? – r.bhardwaj Sep 16 '16 at 17:38
  • @r.bhardwaj Yes but you can limit to main node only. I solved it under: http://stackoverflow.com/questions/41161691/how-to-run-a-celery-worker-with-django-app-scalable-by-aws-elastic-beanstalk/41161692#41161692 – smentek Dec 16 '16 at 09:41