2

I have a set of DAGs that have run from 2011-01-01 to 2018-04-01 on a @monthly schedule. I've done this by setting the start_date and end_date on each of the dags. However, I've now realized I need it to run for one more month. I've updated the end_date to be 2018-05-01, however updating this value has not triggered Airflow start another DagRun.

I've read Airflow: changing the crontab time for a DAG in Airflow, but I do not want to change the Dag id because that would cause me to have to re-run all the months and I just want to add one additional month. Is this possible in Airflow?

tobi6
  • 8,033
  • 6
  • 26
  • 41
Ben Rudolph
  • 2,509
  • 2
  • 19
  • 26

2 Answers2

2

You can manually trigger the dag after you have changed the end_date using CLI:

airflow trigger_dag [-h] [-sd SUBDIR] [-r RUN_ID] [-c CONF] [-e EXEC_DATE] dag_id

More info: https://airflow.apache.org/cli.html#trigger_dag

kaxil
  • 17,706
  • 2
  • 59
  • 78
  • Thanks, will try this. I have dozens of dags to trigger unfortunately. It'd be great to know the thing that is preventing Airflow from picking up these new dags and modify that. – Ben Rudolph May 26 '18 at 21:42
0

It is possible. @monthly is just a nicer way to say "0 0 1 * *" [1], If you have end-date set up,

end_date: if specified, the scheduler won't go beyond this date[2]

You can try use 2018-05-02 instead.

[1]https://airflow.apache.org/scheduler.html#dag-runs [2]https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L2164

Chengzhi
  • 2,531
  • 2
  • 27
  • 41
  • This was the initial thing I tried, and it's not working unforutnately. Since I've already set an end date I seems like Airflow has stored that end date somewhere and is not backfilling any further. – Ben Rudolph May 25 '18 at 19:58
  • Which version of airflow are you using? Also, since you are using @monthly can you try bump up `end_date` to `2018-06-01` – Chengzhi May 25 '18 at 21:55
  • Version 1.9.0. Yes I did try to bump it to 2018-06-01 (not just 1 day) – Ben Rudolph May 25 '18 at 23:58