0

I am trying to copy files from my edge node to HDFS using oozie. Many suggested to setup password less ssh to get this done.

Iam unable to login to oozie user as it is a service user.

Is there any other way other than password less ssh.

Thanks in advance.

sandy kay
  • 115
  • 1
  • 14
  • You need an authentication mechanism when you try to access edgenode. Because your oozie actions run as a map-reduce job on datanodes of your cluster. If it has to fetch files on edgenode, then some authentication mechanism is needed. It could be either with passwordless authentication using private key and public key or using expect command in Linux. I would prefer to do place a password less authentication. – Deepak Janyavula Nov 13 '17 at 13:55
  • Also, if you want to perform passwordless authentication, not always you need to login as oozie user. If you are using oozie ssh action, your oozie launcher executes the ssh command and you need oozie public key and private key to enable password less authentication. If you use oozie shell action, your action will be executed either by yarn user or your Linux user using which your shell action is launched (to know precisely check what is configured in your cluster yarn settings as in this link: https://blog.cloudera.com/blog/2013/03/how-to-use-oozie-shell-and-java-actions/). – Deepak Janyavula Nov 13 '17 at 13:58
  • Thanks for your response. How ever I am still unable to figure out on how to get the oozie public key to enable password less authentication. – sandy kay Nov 19 '17 at 06:39

1 Answers1

0

Other than password less ssh there are two more options : 1. My preferred option : Use JSch java library and create a java application which will accept a shell script to be executed as argument. Using the JSch , it will perform ssh on the configured edge node and execute the shell script on the edge node. In the jsch, you can configure, the edgenode username and password. Use 'JCEKS' file to store the password. Then add a Java Action in Oozie to run the java application created using JSch. 2. Use "/usr/bin/expect" library to create a shell script, which will perform ssh on edgenode and then run the configured shell script. More details are here Use expect in bash script to provide password to SSH command

Amit Kumar
  • 1,544
  • 11
  • 23