Spark submit to yarn as a another user

Question

Is it possible to submit a spark job to a yarn cluster and choose, either with the command line or inside the jar, which user will "own" the job?

The spark-submit will be launch from a script containing the user.

PS: is it still possible if the cluster has a kerberos configuration (and the script a keytab) ?

score 9 · Answer 1 · answered Oct 28 '16 at 17:34

9

For a non-kerberized cluster: export HADOOP_USER_NAME=zorro before submitting the Spark job will do the trick.
Make sure to unset HADOOP_USER_NAME afterwards, if you want to revert to your default credentials in the rest of the shell script (or in your interactive shell session).

For a kerberized cluster, the clean way to impersonate another account without trashing your other jobs/sessions (that probably depend on your default ticket) would be something in this line...

export KRB5CCNAME=FILE:/tmp/krb5cc_$(id -u)_temp_$$
kinit -kt ~/.protectedDir/zorro.keytab zorro@MY.REALM
spark-submit ...........
kdestroy

answered Oct 28 '16 at 17:34

Samson Scharfrichter

8,884
1
17
36

For details, you may look at https://stackoverflow.com/questions/37379765/service-specific-users-not-created-in-cloudera and http://stackoverflow.com/questions/11041253/set-hadoop-system-user-for-client-embedded-in-java-webapp/11062529#11062529 – Samson Scharfrichter Oct 28 '16 at 17:41
this answer is okay, although assumes that someone has access to all keytabs of all users that you might have to run a spark job as... that's not always feasible and always not secure. also managing keytabs might be a nightmare (e.g. when a user changes a password, that keytab has to be changed etc). a better way is to use `proxy authentication` - see my answer below on this. – Tagar Jan 03 '19 at 21:34
`export HADOOP_USER_NAME=hadoop` worked with AWS EMR. – Tharaka Jul 23 '20 at 02:51

score 5 · Answer 2 · answered Mar 21 '17 at 14:49

5

For a non-kerberized cluster you can add a Spark conf as:

--conf spark.yarn.appMasterEnv.HADOOP_USER_NAME=<user_name>

answered Mar 21 '17 at 14:49

Kalyan Ghosh

483
11
12

Did you test that in both `yarn-client` and `yarn-cluster` modes? – Samson Scharfrichter Mar 26 '17 at 21:48
1

This is not working for me, could you add a link to some documentation where this is stated? I cant find it anywhere – Felipe Gonzalez Dec 05 '19 at 14:15

score 3 · Answer 3 · answered Jan 03 '19 at 21:28

3

Another (much safer) approach is to use proxy authentication - basically you create a service account and then allow it to impersonate to other users.

$ spark-submit --help 2>&1 | grep proxy
  --proxy-user NAME           User to impersonate when submitting the application.

Assuming Kerberized / secured cluster.

I mentioned it's much safer because you don't need to store (and manage) keytabs of alll users you will have to impersonate to.

To enable impersonation, there are several settings you'd need to enable on Hadoop side to tell which account(s) can impersonate which users or groups and on which servers. Let's say you have created svc_spark_prd service account/ user.

hadoop.proxyuser.svc_spark_prd.hosts - list of fully-qualified domain names for servers which are allowed to submit impersonated Spark applications. * is allowed but nor recommended for any host.

Also specify either hadoop.proxyuser.svc_spark_prd.users or hadoop.proxyuser.svc_spark_prd.groups to list users or groups that svc_spark_prd is allowed to impersonate. * is allowed but not recommended for any user/group.

Also, check out documentation on proxy authentication.

Apache Livy for example uses this approach to submit Spark jobs on behalf of other end users.

answered Jan 03 '19 at 21:28

Tagar

13,911
6
95
110

this answer is okay, although assumes that the privileged "proxy" account is not usable by anyone at any time -- otherwise you have _no_ authentication. – Samson Scharfrichter Jan 05 '19 at 11:51
that's exactly right. such special service accounts in our organization are pretty much locked up (e.g. they can't have a unix session etc), and we only use them to authenticate in Kerberos such an impersonation service for Spark jobs. vs. your approach assumes that someone has access to all keytabs (or perhaps maybe even to passwords) of all users that you might have to run a spark job as... that's not always feasible and always not secure. Also managing keytabs might be a nightmare (e.g. when a user changes a password, that keytab has to be changed etc). `proxy authentication`is more secure – Tagar Jan 07 '19 at 07:06
Oh, please, stop calling that "my" approach. It's the _"solve your own problem all by yourself without admin privileges"_ approach. Plus, the question is 2 years old -- are you sure Spark supported proxy accounts by then?? – Samson Scharfrichter Jan 07 '19 at 19:59
`proxy-user` was added to Spark in release 1.3.0 almost 4 years ago by this commit - https://github.com/apache/spark/commit/8e75b0ed04788a3873c60544c34e3db0e1e7a373#diff-63a5d817d2d45ae24de577f6a1bd80f9R484 on Feb/2015 :-) – Tagar Jan 07 '19 at 20:46

score 1 · Answer 4 · edited May 23 '17 at 12:25

1

If your user exists, you can still launch your spark submit with su $my_user -c spark submit [...]

I am not sure about the kerberos keytab, but if you make a kinit with this user it should be fine.

If you can't use su because you don't want the password, I invite you to see this stackoverflow answer: how to run script as another user without password

edited May 23 '17 at 12:25

Community

1
1

answered Oct 28 '16 at 09:48

kulssaka

226
8
27

I will not be able to su to another user. The user that will launch spark-submit will be like www-data so su will not be possible and it will node be able to do a kinit as it require the final user password. – Benjamin Oct 28 '16 at 09:53
the one who launch the spark job is the owner. su -c will not change your user, only it will run the job as the user your selected; edit: ok, I modified my post – kulssaka Oct 28 '16 at 09:55

Spark submit to yarn as a another user

4 Answers4

Linked