Running cron jobs on aws elastic beanstalk - django

Question

I'm having trouble getting my cron jobs to execute.

Setup:

Django - 1.9

Elastic beanstalk - 64bit Amazon Linux 2016.03 v2.1.3 running Python 3.4

I've tried doing this a couple of ways so far:

Using a cron.yaml file: Didn't touch anything else - just added a cron.yaml file to my project root folder
```
version: 1
cron:
  - name: "test"
url: "http://website.com/workers/test"
schedule: "*/10 * * * *"
```
Using a container command and a separate cron.txt file:

Added this line in my .ebextensions/development.config file

05_some_cron:
    command: "cat .ebextensions/crontab.txt > /etc/cron.d/crontab && chmod 644 /etc/cron.d/crontab"
    leader_only: true

and in .ebextensions/crontab.txt

*/10 * * * * source /opt/python/run/venv/bin/activate && python mysite/manage.py test

The app deploys successfully in both cases.

Manually (in a browser) going to http://website.com/workers/test has the intended outcome (in the first case).
Adding source /opt/python/run/venv/bin/activate && python mysite/manage.py test as a management command runs the correct script once on deploying.

The logs do not show any GETS on that url.

What am I doing wrong? Am I missing some step of the process or some setup step on EBS?

Also what is the best ways to run cron jobs for django applications hosted on EBS? - django apps can run management commands either from the command line as in attempt 2 or by extending a GET or POST url as in attempt 1.

@PeterBrittain Nope - the logs are the same as a normal deployment without the cron.yaml file — Dustfinger, Aug 01 '16 at 13:58
And have you tried looking at what happens when your commands are running - as described in http://stackoverflow.com/questions/4811738/cron-job-log-how-to-log? — Peter Brittain, Aug 01 '16 at 21:24
cron.yaml in ElasticBeanstalk only sends POST requests to the designated endpoint. — voidnologo, Sep 29 '16 at 19:51
Have you check out this SO answer https://stackoverflow.com/a/28719447/3518108 ? — Sam H., Jun 09 '17 at 05:32
You may be having issues with your I AM roles / security groups. See here https://stackoverflow.com/a/59520061/5403449 — Josh, Jan 19 '20 at 20:05

score 5 · Answer 1 · answered Aug 22 '18 at 06:30

2018 Update using Django 2.0.6 + AWS ElasticBeanstalk + Cronjob

I found I needed to export the AWS environment variables using source /opt/python/current/env to prevent manage.py from throwing the error "The SECRET_KEY setting must not be empty".

This is because I had placed my Django Secret key in the os.environ for beanstalk which it seems is not accessible by shell/cron by default.

See https://forums.aws.amazon.com/thread.jspa?threadID=108465

My final cron job .txt script was as follows. (process_emails is my own django management function, myjob.log is where the output of the function call is logged).

*/15 * * * * source /opt/python/run/venv/bin/activate && cd /opt/python/current/app/ && source /opt/python/current/env && python manage.py process_emails >> /var/log/myjob.log 2>&1

Using Django 2.1.1 as of today (2020 April) with AWS ElasticBeanstalk, I can confirm that this is still a valid solution! — James Yoo, Apr 04 '20 at 00:37

score 3 · Answer 2 · answered Jun 21 '17 at 15:15

3

This works for me in Django 1.10.6 + AWS ElasticBeanstalk + Cronjob

* * * * * source /opt/python/run/venv/bin/activate && cd /opt/python/current/app/ && python manage.py do_your_thing > $HOME/cron-sqs.log 2>&1

do_your_thing is a management command

answered Jun 21 '17 at 15:15

Karl Zillner

581
7
16

Running cron jobs on aws elastic beanstalk - django

2 Answers2

Linked