12

I've got dag_prime and dag_tertiary.

  • dag_prime: Scans through a directory and intends to call dag_tertiary on each one. Currently a PythonOperator.
  • dag_tertiary: Scans through the directory passed to it and does (possibly time-intensive) calculations on the contents thereof.

I can call the secondary one from a system call from the python operator, but i feel like there's got to be a better way. I'd also like to consider queuing the dag_tertiary calls, if there's a simple way to do that. Is there a better way than using system calls?

Thanks!

brendan
  • 310
  • 1
  • 2
  • 9

2 Answers2

29

Use airflow.operators.trigger_dagrun for calling one DAG from another.

The details can be found in operator trigger_dagrun Airflow documentation.

Following post gives a good example of using this operator: https://www.linkedin.com/pulse/airflow-lesson-1-triggerdagrunoperator-siddharth-anand

Noam Hacker
  • 4,671
  • 7
  • 34
  • 55
Him
  • 1,609
  • 12
  • 20
  • 1
    Can this be done in a loop though? I've seen the triggerdagrunoperator and from what i can tell it does what the name says - triggers a (single) dag run. Is there a way to trigger many of those in a loop? – brendan Jul 25 '17 at 14:50
  • 2
    You can place it in a loop and trigger through it as many times as you want. I have used dynamic task creation in a DAG using for loop for my use cases (with different operators but it should be the same with this operator as well). – Him Jul 25 '17 at 16:27
  • Can some parameters be passed from one dag to other dag while calling? if so, how to access those passed parameters in the triggered dag? – SudhakarH Nov 10 '22 at 06:57
0

Use TriggerDagRunOperator from airflow.operators.dagrun_operator and pass the other DAG name to triger_dag_id parameter.

Follow Airflow updated documentation dag_run_operator Airflow Documentation

SudhakarH
  • 541
  • 5
  • 16