3

I'm using ECS for the first time. I have dockerized my Django 2.2 application and using ECS and uwsgi to run the Django application in production.

While in the development environment, I had to run three commands to run Django server, celery and celery beat

python manage.py runserver
celery -A qcg worker -l info
celery beat -A qcg worker -l info

Where qcg is my application.

My Dockerfile has following uwsgi configuration 

EXPOSE 8000
ENV UWSGI_WSGI_FILE=qcg/wsgi.py

ENV UWSGI_HTTP=:8000 UWSGI_MASTER=1 UWSGI_HTTP_AUTO_CHUNKED=1 UWSGI_HTTP_KEEPALIVE=1 UWSGI_LAZY_APPS=1 UWSGI_WSGI_ENV_BEHAVIOR=holy

ENV UWSGI_WORKERS=2 UWSGI_THREADS=4

ENV UWSGI_STATIC_MAP="/static/=/static_cdn/static_root/" UWSGI_STATIC_EXPIRES_URI="/static/.*\.[a-f0-9]{12,}\.(css|js|png|jpg|jpeg|gif|ico|woff|ttf|otf|svg|scss|map|txt) 315360000"

USER ${APP_USER}:${APP_USER}

ENTRYPOINT ["/app/scripts/docker/entrypoint.sh"]

The entrypoint.sh file has

exec "$@"

I have created the ECS task definition and in the container's command input, I have

uwsgi --show-config

This starts the uwsgi server.

Now I'm running 1 EC2 instance in the cluster and running one service with two instances of the task definition.

I couldn't figure out how to run the celery task and celery beat in my application.

Do I need to create separate tasks for running celery and celery-beat?

Anuj TBE
  • 9,198
  • 27
  • 136
  • 285

1 Answers1

1

Yes, you need to run separate ECS tasks (or separate ECS services) for celery beat and celery worker. Celery Beat will send the Celery tasks to the Celery worker.

I use separate Dockerfiles for Celery, Celery beat, and Django.

Worker Dockerfile something like this:

FROM python:3.8

ENV PYTHONUNBUFFERED 1
ADD requirements.txt /requirements.txt

RUN pip install -r /requirements.txt
ADD . /src
WORKDIR /src

CMD ["celery", "-A", "<appname>", "worker"]

and Beat Dockerfile:

FROM python:3.8

ENV PYTHONUNBUFFERED 1
ADD requirements.txt /requirements.txt

RUN pip install -r /requirements.txt
ADD . /src
WORKDIR /src

CMD ["celery", "-A", "<appname>", "beat"]
Weston
  • 11
  • 2