How to kill a running Spark application?

Question

I have a running Spark application where it occupies all the cores where my other applications won't be allocated any resource.

I did some quick research and people suggested using YARN kill or /bin/spark-class to kill the command. However, I am using CDH version and /bin/spark-class doesn't even exist at all, YARN kill application doesn't work either.

enter image description here

Can anyone with me with this?

if you are in a test env : ps aux | grep spark -> get the pid of spark and kill it from command line — eliasah, Apr 10 '15 at 15:54
@eliasah Yeah... a job in production got hung due the failure in one host. — B.Mr.W., Apr 10 '15 at 16:04

score 271 · Accepted Answer · edited Jun 17 '21 at 08:53

271

copy paste the application Id from the spark scheduler, for instance application_1428487296152_25597
connect to the server that have launch the job
yarn application -kill application_1428487296152_25597

edited Jun 17 '21 at 08:53

yatu

86,083
12
84
139

answered May 15 '15 at 06:55

Gérald Reinhart

3,101
1
13
14

1

How do you get to the spark scheduler? – makansij Jul 11 '16 at 16:48
Is it the same as the `web UI` ? – makansij Jul 14 '16 at 18:47
@Hunle You can get the ID from `Spark History UI` or YARN `RUNNING` apps UI (http://yarn-host:8088/cluster/apps/RUNNING) or from `Spark Job Web UI` URL (http://yarn-host:8088/proxy/application__) – CᴴᴀZ Feb 22 '17 at 06:49
3

can one kill several at once : yarn application -kill application_1428487296152_25597 application_1428487296152_25598 ... ?? – DataYoda May 04 '17 at 06:16

score 14 · Answer 2 · answered Sep 22 '20 at 15:46

14

First use:

yarn application -list

Note down the application id Then to kill use:

yarn application -kill application_id

answered Sep 22 '20 at 15:46

Ankit Anand

321
3
7

2

For future me, one command to combine both of these `yarn application -list|cut -f 1 |grep "application_" |xargs -I {} -P 1 -n 1 yarn application -kill {}` – Shmuel Kamensky Dec 14 '21 at 14:16

score 12 · Answer 3 · edited May 08 '19 at 08:14

It may be time consuming to get all the application Ids from YARN and kill them one by one. You can use a Bash for loop to accomplish this repetitive task quickly and more efficiently as shown below:

Kill all applications on YARN which are in ACCEPTED state:

for x in $(yarn application -list -appStates ACCEPTED | awk 'NR > 2 { print $1 }'); do yarn application -kill $x; done

Kill all applications on YARN which are in RUNNING state:

for x in $(yarn application -list -appStates RUNNING | awk 'NR > 2 { print $1 }'); do yarn application -kill $x; done

score 3 · Answer 4 · answered Oct 11 '17 at 02:42

3

https://hadoop.apache.org/docs/stable/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html#Cluster_Application_State_API

PUT http://{rm http address:port}/ws/v1/cluster/apps/{appid}/state

{
  "state":"KILLED"
}

answered Oct 11 '17 at 02:42

Starwalker

96
10

score 1 · Answer 5 · answered Feb 19 '20 at 05:20

This might not be an ethical and preferred solution but it helps in environments where you can't access the console to kill the job using yarn application command.

Steps are

Go to application master page of spark job. Click on the jobs section. Click on the active job's active stage. You will see "kill" button right next to the active stage.

This works if the succeeding stages are dependent on the currently running stage. Though it marks job as " Killed By User"

How to kill a running Spark application?

5 Answers5

Linked