0

I am using Airflow 2.5.1 running on top of AKS, I am facing issues where the tasks in running DAGs are stuck in queue and not starting.

stalled_task_timeout = 600 is implemented but it does not seem working.

Could you please help and please let me know if you faced similar issues and how do you resolved it?

To have all tasks in DAGs running as per expected

kh0000
  • 1
  • 1
  • there are many different reasons why this happens. could you share more details - logs from scheduler, your worker setup? – arunvelsriram Jul 21 '23 at 04:53
  • check this as well: https://stackoverflow.com/questions/45853013/airflow-tasks-get-stuck-at-queued-status-and-never-gets-running – arunvelsriram Jul 21 '23 at 04:54

2 Answers2

0

Picture of airflow dagIt can happen if your dags are paused. You can try running after turning on your dag.

Rishav Jain
  • 21
  • 3
  • hi there, the dag itself is running, but the task under the dag is stuck in queue and not starting nor running – kh0000 Jul 21 '23 at 06:22
0

This GitHub Issue partially explains why tasks can get stuck in queued for longer than stalled_task_timeout.

stalled_task_timeout has been replaced by AIRFLOW__SCHEDULER__TASK_QUEUED_TIMEOUT starting in Airflow 2.6.0, which resolves this problem and reliably handles tasks that get stuck in the queued state.

RNHTTR
  • 2,235
  • 2
  • 15
  • 30