2

I'm trying to install Hadoop on my laptop. I followed this guide: https://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

When I try to run start-all.sh I get this:

vava@vava-ThinkPad:/usr/local/hadoop-3.1.1/sbin$ bash start-all.sh
WARNING: Attempting to start all Apache Hadoop daemons as vava in 10 seconds.

WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [localhost]
pdsh@vava-ThinkPad: localhost: rcmd: socket: Permission denied
Starting datanodes
pdsh@vava-ThinkPad: localhost: rcmd: socket: Permission denied
Starting secondary namenodes [vava-ThinkPad]
pdsh@vava-ThinkPad: vava-ThinkPad: rcmd: socket: Permission denied
Starting resourcemanager
resourcemanager is running as process 3748.  Stop it first.
Starting nodemanagers
pdsh@vava-ThinkPad: localhost: rcmd: socket: Permission denied

I tried to follow this questions but nothing changed:

starting hadoop process using start-all.sh runs into issues

Hadoop permission issue

EDIT : After I tried all the options, the only one that seems to work is export PDSH_RCMD_TYPE=ssh. Now the problem is with namenode and datanode. It doesn't start properly:

vava@vava-ThinkPad:/usr/local/hadoop-3.1.1$ sbin/start-all.sh
WARNING: Attempting to start all Apache Hadoop daemons as vava in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [localhost]
pdsh@vava-ThinkPad: localhost: ssh exited with exit code 1
Starting datanodes
localhost: ERROR: Cannot set priority of datanode process 10937
pdsh@vava-ThinkPad: localhost: ssh exited with exit code 1
Starting secondary namenodes [vava-ThinkPad]
Starting resourcemanager
Starting nodemanagers
Luca Vavassori
  • 31
  • 1
  • 6
  • 1
    Did you try with sudo? sudo start-all.sh – DeshDeep Singh Sep 10 '18 at 13:22
  • If you `ssh localhost`, does it work without a password prompt? Secondly, that tutorial is super old. Please, please follow the official Hadoop documentation for your specific version – OneCricketeer Sep 10 '18 at 14:20
  • @DeshDeepSingh yes, nothing changed. – Luca Vavassori Sep 11 '18 at 13:51
  • According to this, whatever configurations you took from that tutorial are not going to work with Hadoop3, as I mentioned before https://stackoverflow.com/questions/46283634/localhost-error-cannot-set-priority-of-datanode-process-32156 – OneCricketeer Sep 12 '18 at 03:01

3 Answers3

0

I would check:

  • export PDSH_RCMD_TYPE=ssh in your terminal
  • Local Firewall settings
  • Running the command as root: sudo /usr/local/hadoop-3.1.1/sbin$ bash start-all.sh
  • chmod -R 755 /usr/local/hadoop-3.1.1

For your additional question:

  • Set JAVA_HOME in hadoop-env.sh and make sure all other options are correct in this file
  • Change your user, Attempting to start all Apache Hadoop daemons as vava in 10 seconds. "vava" Is wrong, try su -l hdfs then run the script
Petro
  • 3,484
  • 3
  • 32
  • 59
  • 'export PDSH_RCMD_TYPE=ssh' seems to work, but now I have a problem with datanode and namenode. I edited the original question. Thank @Petro for having solved the main problem. – Luca Vavassori Sep 11 '18 at 17:31
  • Change your user: `Attempting to start all Apache Hadoop daemons as vava in 10 seconds.` try `su -l hdfs` then run the script...see updated answer – Petro Sep 12 '18 at 14:16
0

Create a new file 

/etc/pdsh/rcmd_default

write ”ssh“ to it, and then save&quit. Make sure you enter a return character and start a new line, otherwise SSH exit with code 1 will be prompted

echo "ssh" > /etc/pdsh/rcmd_default
HbnKing
  • 1,762
  • 1
  • 11
  • 25
0

In my case you need to make sure to copy the RSA to the current localhost

ssh-copy-id -i /home/hadoop/.ssh/id_rsa.pub hadoop@localhost

Assuming you are logged in to the node-master with "hadoop"

MMSA
  • 810
  • 8
  • 22