Sqoop Import --password-file function not working properly in sqoop 1.4.4

Question

I am using hadoop-1.2.1 and sqoop version is 1.4.4.

I am trying to run the following query.

sqoop import --connect jdbc:mysql://IP:3306/database_name --table clients --target-dir /data/clients --username root --password-file /sqoop.password -m 1

sqoop.password is a file which is kept on HDFS in path /sqoop.password with permission 400.

It is giving me an error

Access denied for user 'root'@'IP' (using password: YES)

Can anyone provide solution for this? Thanks in advance.

score 17 · Accepted Answer · answered Apr 24 '16 at 01:21

17

"\n" is being written in file when you vi the file and write the password. Better use the below approach to avoid problems

echo -n "Your_sqoop_password" > sqoop.password

answered Apr 24 '16 at 01:21

Nandakishore

981
1
9
22

2

Exactly. the password should be naked, no newline. – ChuckCottrill Feb 02 '17 at 01:22
1

Or use vim as per https://stackoverflow.com/questions/1050640/vim-disable-automatic-newline-at-end-of-file#16114535, since part of the point of using a password file is to avoid the password appearing wherever command line usages appear in logs or `ps`. – Philip Adler Jan 16 '18 at 13:22
Perfect, this fixed my issue. – Kynrek Dec 23 '18 at 22:38

score 14 · Answer 2 · answered Aug 12 '15 at 22:44

14

Not sure if you are still having this issue. The password file can be in any folder. Try the following syntax and it should work:

--password-file file:///user/root/database.password

answered Aug 12 '15 at 22:44

ganeshrj

141
1
6

Thanks a lot. I had the same problem and got solved with this answer. – SrinR Feb 05 '16 at 10:28
1

Thanks, and it worked for me as well. I would add that if you want it to look in hdfs, for me I had to use the following: `--password-file hdfs:///user/root/database.password` – Shanemeister Apr 20 '18 at 17:28
I was getting an error that the provided password file did not exist. But adding file:/// as per this answer helped me solve that error – user131476 Oct 06 '20 at 07:04

Prasad Khode · Answer 3 · 2017-02-02T06:28:27.253

3

As per the sqoop documentation

You should save the password in a file on the users home directory with 400 permissions and specify the path to that file using the --password-file argument, and is the preferred method of entering credentials. Sqoop will then read the password from the file and pass it to the MapReduce cluster using secure means with out exposing the password in the job configuration. The file containing the password can either be on the Local FS or HDFS.

If I am running my sqoop job with root user then my password file will be in /user/root/ in HDFS

sqoop import --connect jdbc:mysql://database.example.com/employees \
    --username venkatesh --password-file /user/root/database.password

For more details you can check this

edited Feb 02 '17 at 06:28

answered Apr 17 '15 at 12:06

Prasad Khode

6,602
11
44
59

Hi Prasad, Didn't work on my end. It is giving the same error. – Kanav Narula Apr 20 '15 at 07:02
I had to place the password file into HDFS to get it to work. sigh. – ChuckCottrill Feb 02 '17 at 01:23
yes as per the documentation that I have added above `The file containing the password can either be on the Local FS or HDFS.` and im my case I have placed the password file in `HDFS` – Prasad Khode Feb 02 '17 at 06:26

score 3 · Answer 4 · edited Feb 02 '17 at 06:29

3

While creating password, use echo -n option. (-n option removes all trailing spaces).
Suppose you have a password "myPassword" and you want to save it to a file sqoop.password, then follow below steps:

Create password using command
```
echo -n "myPassword" > sqoop.password
```
Upload the file to HDFS as the file needs to be present in HDFS
```
hadoop fs -put sqoop.password /user/keepMyFilesHere
```

Write the scoop import command

sqoop list-tables --connect jdbc:mysql://localhost/kpdatabase --username root --password-file /user/karanpreet.singh/sqoop.password

This will definitely work!

edited Feb 02 '17 at 06:29

Prasad Khode

6,602
11
44
59

answered Aug 16 '16 at 11:22

KP Fingh

79
2

The subtlety that the password file must be in hadoop is missing from other answers, so this answer is more complete! – ChuckCottrill Nov 30 '16 at 19:52
Using echo is not advised for shared environments since it makes the password visible to other users. – Philip Adler Jan 16 '18 at 13:23

Ayub · Answer 5 · 2015-09-16T11:03:04.207

2

Check if there is any garbage character in your password file. I was getting the same issue and finally found that the file contained a \n character at the end and sqoop considered that also as a part of the password string. Try creating a password file as mentioned below and than use the password file : echo -n "root_password" > password.txt Place your password in place of root_password.

edited Sep 16 '15 at 11:03

answered Sep 16 '15 at 10:52

Ayub

21
2

On linux, ls -l, or wc will disclose the file size, and od -c or od -c (do you prefer character or hex output?) will show you the contents. – ChuckCottrill Nov 30 '16 at 19:51

score 1 · Answer 6 · answered Aug 13 '16 at 21:27

Verify that the user running the sqoop command is the owner of the file /sqoop.password (which has the 400 permissions set). This will ensure that the password file is readable.
Make sure that the sqoop.password file does not have any extra non-printing characters at the end (for example, a newline character). One easy way to check this is to look at the size of the file. If the password is say 'sqoop', the size of the file should be 5. If it is larger, delete the sqoop.password file from HDFS, regenerate it locally and put it back on HDFS.

The command that should report the above information pieces is:

hadoop fs -ls /sqoop.password

score 0 · Answer 7 · answered Nov 13 '19 at 18:36

0

please follow below link for all your sqoop password related doubts.

www.ericlin.me

answered Nov 13 '19 at 18:36

Pala

2,011
3
15
17

Sqoop Import --password-file function not working properly in sqoop 1.4.4

7 Answers7

Linked