NodeManager and ResourceManager processes do not start

Question

I am setting up a Multi-Node cluster and my NodeManager and ResourceManager processes are not starting for some reason and I can't figure out why. When I run the jps command, I only see the NameNode and SecondaryNameNode and JPS processes. As a result, my MapReduce job won't work. This is my configuration

yarn-site.xml - across NameNode and DataNodes

<configuration>
<property>
  <name>yarn.nodemanager.aux-services</name>
  <value>mapreduce_shuffle</value>
</property>
<property>
  <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
  <value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
<property>
    <name>yarn.resourcemanager.hostname</name>
    <value>ec2PathToMyNameNode.compute-1.amazonaws.com</value>
  </property>
</configuration>

And my hosts file is this on the NameNode:

nameNodeIP nameNodePublicDNS.compute-1.amazonaws.com
dataNode1IP dataNode1PublicDNS.compute-1.amazonaws.com
dataNode2IP dataNode2PublicDNS.compute-1.amazonaws.com
dataNode3IP dataNode3PublicDNS.compute-1.amazonaws.com
127.0.0.1 localhost

When I run my MapReduce job it says it's unable to connect at 8032. I am using Hadoop 3.1.2

Edit:

I Checked the logs and i found the following exception:

Caused by: java.lang.ClassNotFoundException: javax.activation.DataSource at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583) at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:190) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:499) ... 83 more

Error injecting constructor, java.lang.NoClassDefFoundError: javax/activation/DataSource at org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver.(JAXBContextResolver.java:41) at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebApp.setup(RMWebApp.java:54) while locating org.apache.hadoop.yarn.server.resourcemanager.webapp.JAXBContextResolver

1 error at com.google.inject.internal.InjectorImpl$2.get(InjectorImpl.java:1025) at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1051) at com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory$GuiceInstantiatedComponentProvider.getInstance(GuiceComponentProviderFactory.java:345)

what is the command used to start hadoop; start-dfs.sh or start-all.sh? — Hadi, Mar 03 '19 at 19:27
what is the log directory you specified? check the `HADOOP_LOG_DIR` global variable value — Hadi, Mar 03 '19 at 19:34
You have to check the Hadoop log directory, it contains the errors thrown by Nodemanager and resourcemanager when they try to start — Hadi, Mar 03 '19 at 19:35
I only set HADOOP_CONF_DIR and HADOOP_HOME. Do I need to execute the formatting of hdfs in a specific directory? — Namorange, Mar 03 '19 at 19:35
Try to search for these exceptions, i found some links that can help: (1) https://issues.apache.org/jira/browse/HADOOP-14978 (2) https://jira.apache.org/jira/browse/YARN-8802 — Hadi, Mar 03 '19 at 19:47
Thanks, you helped me figure it out. I wouldn't have thought about the logs. — Namorange, Mar 03 '19 at 19:57
I will try to sum all what we mentioned in the comments inside an well formatted answer so other users will benefit from it easly, and you have to write your own answer when you solves the issue. — Hadi, Mar 03 '19 at 20:00
check the third suggestion in my answer, i didn't mention it in the comments, also check the links i provided — Hadi, Mar 03 '19 at 20:55

score 2 · Answer 1 · answered Mar 03 '19 at 20:11

Trying to figure out the issue

(1) Start-dfs.sh vs Start-all.sh

Check that you are using Start-all.sh command when you are trying to start hadoop because Start-dfs.sh will only start the namenode and datanodes

(2) Check the Hadoop logs

Check for the HADOOP_LOG_DIR global variable value to get the Log dir, because it will include all exception thrown when trying to start the Namenode Manager and the Resource Manager

(3) Check for the installed Java version

The error may be thrown by an incompatible Java version, check that you have installed the latest Java version.

(4) Check Hadoop Common issues

Based on the error you provided in the answer update you may find these issue links relevant:

More information

For more information you can check my article on Medium, it may give you some insights:

Installing Hadoop 3.1.0 multi-node cluster on Ubuntu 16.04 Step by Step

score 1 · Answer 2 · answered Jul 31 '20 at 09:42

1

my problem is that I used java11 to cooperate with hadoop.

so what i do is

1.rm /Library/Java/*

2.download java8 from https://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html

3.install java8jdk and

4.fix the JAVA_HOME in hadoop-env.sh

5.stop-all.sh

6.start-dfs.sh

7.start-yarn.sh

answered Jul 31 '20 at 09:42

Юрий Скворцов

11
1

More explanation on what each step does or what its purpose would be tremendously helpful! – Jenea Vranceanu Jul 31 '20 at 10:14

score -1 · Answer 3 · answered Jul 30 '19 at 19:15

-1

[pdash@localhost hadoop]$ export YARN_RESOURCEMANAGER_OPTS="--add-modules=ALL-SYSTEM" [pdash@localhost hadoop]$ export YARN_NODEMANAGER_OPTS="--add-modules=ALL-SYSTEM"

It will work for sure I tried from apache JIRA log ....Thank PRAFUL

answered Jul 30 '19 at 19:15

PRAFULLA KUMAR DASH

33
4

NodeManager and ResourceManager processes do not start

3 Answers3

Trying to figure out the issue

More information