2

I am working in Ariflow 2.2.3 and I can't figure out how to trigger my dag with a past execution date. When I click Trigger dag with Config, I changed the calendar to the date I wanted but when I clicked run, I saw the run but it didn't run.

I also tried putting the date in the config section with {"start_date":"date"} but that didn't work aswell.

Any idea how to trigger a dag with a date in the past?

Maggie
  • 41
  • 1
  • 8

1 Answers1

0

To create a past Airflow run you have multiple option, but most of them needs to update the start date of your dag to be older than the date of the desired run date (first four options), otherwise the run will be marked as succeeded without being executed.

  1. Via Airflow UI: you can click on the run icon, and choose trigger DAG w/ config, then choose the logical date you want, and create the run.
  2. Via Airflow REST API: here is the doc
  3. Via Airflow python lib: 
    DagBag(read_dags_from_db=True).get_dag(<your dag name>).create_dagrun(
    run_id=<run id>,
    run_type=DagRunType.MANUAL,
    execution_date=<execution date>,
    state=State.QUEUED,
    external_trigger=True,
    data_interval=(<desired start date>, <desired end date>),
    conf={<any conf>: <any value>, ...},
)
  4. Via Airflow CLI: here is the doc 
    airflow dags trigger -e <execution date> <dag id>
  5. Via Airlfow CLI backfill command: here is the doc. It doesn't need an old start date, where the run and the task are managed by the CLI and not the Airflow scheduler. 
    airflow dags backfill -s <start date> -e <end date> <dag id>
    P.S: this concept is based on schedule interval, so if you want a signle run, should be + , otherwise you will have multiple runs between the two days. This is different in the option 3 which create a single run with the and you provided.
Hussein Awala
  • 4,285
  • 2
  • 9
  • 23
  • When I tried choosing the date in the UI with the calendar icon, It had the run as success but all the tasks underneath said no_status. Is there a way to get it actually run? – Maggie Nov 22 '22 at 15:10
  • I just tried the #5 option listed above (under Airflow 2.6.0) but it doesn't work if the given "start date" is earlier than the `start_date` specified in the DAG. The error msg says "No run dates were found for the given dates and dag interval." – Zach May 15 '23 at 16:34