I have followed this tutorial to setup Hadoop 2.2.0 multi-node cluster on Amazon EC2. I have had a number of issues with ssh and scp which i was either able to resolve or workaround with help of articles on Stackoverflow but unfortunately, i could not resolve the latest problem.
I am attaching the core configuration files core-site.xml, hdfs-site.xml etc. Also attaching a log file which is a dump output when i run the start-dfs.sh command. It is the final step for starting the cluster and it is giving a mix of errors and i don't have a clue what to do with them.
So i have 4 nodes exactly the same AMI is used. Ubuntu 12.04 64 bit t2.micro 8GB instances. Namenode SecondaryNode (SNN) Slave1 Slave2
The configuration is almost the same as suggested in the tutorial mentioned above.
I have been able to connect with WinSCP and ssh from one instance to the other. Have copied all the configuration files, masters, slaves and .pem files for security purposes and the instances seem to be accessible from one another.
If someone could please look at the log, config files, .bashrc file and let me know what am i doing wrong.
Same security group HadoopEC2SecurityGroup is used for all the instances. All TCP traffic is allowed and ssh port is open. Screenshot in the zipped folder attached. I am able to ssh from Namenode to secondary namenode (SSN). Same goes for slaves as well which means that ssh is working but when i start the hdfs every thing goes down. The error log is not throwing any useful exceptions either. All the files and screenshots can be found as zipped folder here.
Excerpt from error output on console looks like
Starting namenodes on [OpenJDK 64-Bit Server VM warning: You have loaded library /usr/local/hadoop/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'. ec2-54-72-106-167.eu-west-1.compute.amazonaws.com] You: ssh: Could not resolve hostname you: Name or service not known have: ssh: Could not resolve hostname have: Name or service not known loaded: ssh: Could not resolve hostname loaded: Name or service not known VM: ssh: Could not resolve hostname vm: Name or service not known library: ssh: Could not resolve hostname library: Name or service not known Server: ssh: Could not resolve hostname server: Name or service not known warning:: ssh: Could not resolve hostname warning:: Name or service not known which: ssh: Could not resolve hostname which: Name or service not known guard.: ssh: Could not resolve hostname guard.: Name or service not known have: ssh: Could not resolve hostname have: Name or service not known might: ssh: Could not resolve hostname might: Name or service not known
.....
