3

I'm working on a usecase where I am triggering dags using TriggerDagRunOperator. The idea is that I want to assign priorities in some cases, and in rest cases I want DAG to assume default priorities.

Is it possible to assign priority weight at run time to target DAG using TriggerDagRunOperator?

PS: I'm using Airflow 1.9.

rishm.msc
  • 361
  • 5
  • 16
  • Yes, see my answer here: https://stackoverflow.com/a/58305943/7343742 Similar problem, same solution. – trejas Oct 09 '19 at 14:05
  • @trejas that post is about creating tasks and dags dynamically. My ask here is to assign priority to tasks at runtime to a TaskInstance. – rishm.msc Oct 11 '19 at 20:12
  • Disregard my previous comment: Thinking through your question, it is interesting. I am sure you should be able to do it...Going to give it a go this week sometime. Were you able to make this work? – trejas Oct 15 '19 at 22:12
  • Nope not yet. But, after reading it more about the scheduler and priority weights, I'm not sure if this could be done in Airflow 1.9. – rishm.msc Oct 17 '19 at 05:32
  • Trying to solve this same problem myself and running into issues. Seems `priority_weight` cannot be filled using the template engine. I found out you can subclass an operator and override the `priority_weight_total` property method (which is called by the `base_executor` when queueing task instances to produce the final `priority` value). If there was a way to inject the context into this function that might solve the problem, but I am stumped as to how to do that. – Mason McGough Feb 19 '20 at 18:16
  • I am also looking for the solution for this, How I can assign priority weight at runtime from conf configuration – Rohit Kharat Apr 20 '22 at 12:48

0 Answers0