elastic beanstalk cron run twice

Question

I have one application on elastic beanstalk and cron jobs for it.

The code of setting cron is

container_commands:
  01_some_cron_job:
    command: "echo '*/5 * * * * wget -O - -q -t 1 http://site.com/cronscript/' | crontab"
  leader_only: true

This script calls the mail sender. And I'm receive two message per time.

code of http://site.com/cronscript/ looks like (php code)

require_once('ses.php');
$ses = new SimpleEmailService(EMAIL_SHORTKEY, EMAIL_LONGKEY);
$m = new SimpleEmailServiceMessage();
$m->addTo('user@domain.com');
$m->setFrom('response_service@domain.com');
$m->setSubject('test message');
$m->setMessageFromString('', 'message content');
$send_emails=($ses->sendEmail($m));

When I call http://site.com/cronscript/ from browser's address bar, I receive one message as I want.

score 5 · Answer 1 · answered Jan 23 '14 at 03:57

I believe what's happening is that first time you deploy your app, AutoScaling picks one instance to be a leader and new cron job is created on that instance. Next time you deploy your app, AutoScaling picks another instance to be a leader. So you end up with the same cron job on two instances.

So the basic test would be to ssh to all instances and check their crontab contents with crontab -l

You can avoid duplicate cron jobs by removing old cron jobs on the instance regardless whether it is a leader or not.

container_commands:
  00_remove_old_cron_jobs:
    command: "crontab -r || exit 0"
  01_some_cron_job:
    command: "echo '*/5 * * * * wget -O - -q -t 1 http://example.com/cronscript/' | crontab"
    leader_only: true

As mentioned in Running Cron In Elastic Beanstalk Auto-Scaling Environment: || exit 0 is mandatory because if there is no crontab in the machine the crontab -r command will return a status code > 0 (an error). Elastic Beanstalk stop the deploy process if one of the container_commands fail.

Although I personally never experienced a situation when crontab was not found on Elastic Beanstalk Instance.

You can run /opt/elasticbeanstalk/bin/leader-test.sh to test whether it is a leader instance or not.

Hope it helps.

Here I have posted a solution to the scaling problem (and other potential problems as well): https://stackoverflow.com/a/55798448/2404541 — TheStoryCoder, Apr 22 '19 at 18:58

score 1 · Answer 2 · answered Jun 08 '20 at 14:21

I had the same problem, only change the user from root to the one which I'm logging in with eb ssh and it works. My code looks like this.

files:
"/etc/cron.d/mycron":
    mode: "000644"
    owner: ec2-user
    group: ec2-user
    content: |
        30 1 * * * echo $(date) >> /tmp/cron_start.log; /usr/local/bin/daily_script.sh >> /tmp/crons.log 2>&1;



"/usr/local/bin/daily_script.sh":
    mode: "000755"
    owner: ec2-user
    group: ec2-user
    content: |
        #!/bin/bash

        date > /tmp/date
        # Your actual script content
        /opt/python/run/venv/bin/python3 /opt/python/current/app/cronjob_files/email_data.py >> /opt/python/current/app/cron.logs 2>&1

        exit 0

...

elastic beanstalk cron run twice

2 Answers2