How to stop/kill Airflow tasks from the UI

Question

How can I stop/kill a running task on Airflow UI? I am using LocalExecutor. Even if I use CeleryExecutor, how do can I kill/stop the running task?

score 35 · Answer 1 · edited Sep 26 '18 at 12:21

35

In the DAGs screen you can see the running tasks:

Example

On 'Recent Tasks' press the running icon and Airflow will automatically run the search query with the filters for the Dag Id and State equal to 'running' and show the results on the Task Instances screen (you can find it manually on the tab Browse > Task Instances).

There you can select the presented tasks and set them to another state or delete them.

Please notice that if the DAG is currently running, the Airflow scheduler will start again the tasks you delete. So either you stop the DAG first by changing its state or stop the scheduler (if you are running on a test environment).

edited Sep 26 '18 at 12:21

NoobEditor

15,563
19
81
112

answered Apr 26 '17 at 13:18

Jorge Almeida

387
2
4

2

Will it not restart the task even if i stop the scheduler and delete the task? – Chetan J Apr 27 '17 at 05:08
1

If you stop the scheduler and you do not want that task to run again, just set the task manually to the state "success" instead of deleting it. If you just delete it, the scheduler will run that task again. – Jorge Almeida Apr 28 '17 at 14:09
32

I paused the DAG and deleted the records but the processes are still running. Is there a way to not only change the state according to the executor but also kill the underlying process? – stefanobaghino Jul 28 '17 at 12:44

score 22 · Answer 2 · answered Apr 11 '21 at 08:07

22

Set task to failed state:

Click task
Set task to "Failed" state
All subsequent tasks (if there are any) will also be marked as failed:

answered Apr 11 '21 at 08:07

Hrvoje

13,566
7
90
104

score 14 · Answer 3 · answered Sep 18 '19 at 02:19

14

Simply set the task to failed state will stop the running task.

[2019-09-17 23:53:28,040] {logging_mixin.py:82} INFO - [2019-09-17 23:53:28,039] {jobs.py:2695} WARNING - State of this instance has been externally set to failed. Taking the poison pill.
[2019-09-17 23:53:28,041] {helpers.py:240} INFO - Sending Signals.SIGTERM to GPID 20977

answered Sep 18 '19 at 02:19

kk17

601
6
14

2

This is the solution that worked for me. For others finding this post, more info can be found here: https://issues.apache.org/jira/browse/AIRFLOW-2285 – lampShadesDrifter Oct 09 '19 at 23:39
Works for me also. Other answers imply non-deterministic, but you can check in the logs after changing the state. – jabberwocky Jul 18 '20 at 13:21
This isn't the best plan, if the task has any retries left, it will keep retrying. – szeitlin Sep 02 '20 at 15:48

Pablo · Answer 4 · 2022-01-06T15:36:15.027

11

from airflow gitter (@villasv)

" Not gracefully, no. You can stop a dag (unmark as running) and clear the tasks states or even delete them in the UI. The actual running tasks in the executor won't stop, but might be killed if the executor realizes that it's not in the database anymore. "

edited Jan 06 '22 at 15:36

answered Jun 05 '18 at 19:55

Pablo

3,135
4
27
43

score 7 · Answer 5 · answered Nov 21 '18 at 09:38

As menioned by Pablo and Jorge pausing the Dag will not stop the task from being executed if the execution already started. However there is a way to stop a running task from the UI but it's a bit hacky.

When the task is on running state you can click on CLEAR this will call job.kill() the task will be set to shut_down and moved to up_for_retry immediately hence it is stopped.

Clearly Airflow did not meant for you to clear tasks in Running state however since Airflow did not disable it either you can use it as I suggested. Airflow meant CLEAR to be used with failed, up_for_retry etc... Maybe in the future the community will use this bug(?) and implement this as a functionality with "shut down task" button.