1

I am working with Airflow and having issues for scheduling my DAG's. I have a DAG which I want to run at every Sunday at 00:00 and here is my code

args = {
'owner': 'rsabbir',
'depends_on_past': False,
'start_date': datetime(2018, 2, 22, 00, 00),
'retry_delay': timedelta(minutes=1)
}

dag = DAG(
    dag_id='DAG_column_encoding', default_args=args, schedule_interval='* * * * 0')

I've tried with a different 'start_date' from last week but the task didn't run during weekend. There is also another DAG which I want to run at a particular time(4:30 AM) everyday and I have this code.

args = {
'owner': 'rsabbir',
'depends_on_past': False,
'start_date': datetime(2018, 2, 21, 00, 00),
'retry_delay': timedelta(minutes=1)
}

dag = DAG(
    dag_id='DAG_vacuum_tables', default_args=args, schedule_interval='30 4 * * */1')

Both of them are not working as expected. I have gone through the Airflow documentation but haven't understood how these different cases fo scheduling work.

Can anyone give me a brief idea about how these 'start_date' and 'schedule_interval' actually work together in airflow? Is there any other effective way to work with these two parameters in airflow?

Rafiul Sabbir
  • 626
  • 6
  • 21
  • The schedule_interval in your first example is incorrect it should be 0 0 * * 0 – Blakey Feb 22 '18 at 13:18
  • 1
    Also in your second example you should set your start date to include the 4:30 i.e. datetime(2018, 2, 21, 4, 30) – Blakey Feb 22 '18 at 13:20
  • 1
    As @Blakey suggest, you can use some crontab tool to check if you are not sure, for example`https://crontab.guru/#0_0_*_*_0`, start_date can be without time, 'start_date': datetime(2018, 2, 21), if you have schedule_interval, you can ignore time in start_date. – Chengzhi Feb 22 '18 at 15:59
  • Thanks for the help. Now I have the start_date = datetime(2018, 2, 23) and schedule_interval='40 11 * * 1' so it was supposed to start at 11:30 AM on Monday but it didn't start. Can you tell me what can be the possible problem for not running the DAG at scheduled time? – Rafiul Sabbir Feb 26 '18 at 12:55
  • For the not running DAGs you might want to check this https://stackoverflow.com/questions/49021055/airflow-1-9-0-is-queuing-but-not-launching-tasks – tobi6 Mar 01 '18 at 10:43

0 Answers0