I have a dag that triggers an external DAG using TriggerDagOperator
. The trigger DAG queries a database and based on a type ID, it will trigger the associated external DAG along with the parameters needed for the external DAG. I would want to pass a pool name as part of these parameters and just wondering if I can create the pool in the (external) DAG if it does not exists.
Asked
Active
Viewed 357 times
0

alltej
- 6,787
- 10
- 46
- 87
-
Check this out if it helps https://www.astronomer.io/guides/airflow-pools/ – Smaurya Jun 16 '22 at 14:24
-
That is just straightforward use of pool which we are already using. I am looking for dynamically creating a pool inside DAG. – alltej Jun 16 '22 at 14:27
-
Is this something that you are looking for https://stackoverflow.com/questions/52426489/create-dynamic-pool-in-airflow – Smaurya Jun 16 '22 at 14:31
-
But the problem is pool cannot be templated (`pool="{{dag_run.conf['pool_name']}}"`). So cannot really create the pool because cannot access the conf value. – alltej Jun 16 '22 at 18:49