0

I know that the similar question was asked Applications not shown in yarn UI when running mapreduce hadoop job? but the answers did not solve my problems.

I am running Hadoop streaming on Linux 17.01. I setup a cluster with 3 nodes and 1 master node.

When I start Hadoop, I can access localhost:50070 to see other nodes (all nodes are alive). However, I see no information in "Application" of localhost:8088

enter image description here as well as by command "yarn application -list -appStates ALL". enter image description here

Here is my configuration. My yarn-site.xml (for all nodes)

enter image description here

Here is all processes on master node

enter image description here

The problems may due to yarn services are running on ipv6. However, I followed I followed this thread https://askubuntu.com/questions/440649/how-to-disable-ipv6-in-ubuntu-14-04 to change all Yarn services to ipv4. However, still there is no tasks displayed on Yarn UI, even I can see all nodes in my cluster marked as "active" on Yarn UI.

So, I do not know why this happened. Do you have any suggestion?

Thank you very much.

Community
  • 1
  • 1
cdt
  • 85
  • 10

1 Answers1

0

I haven't typically seen YARN being configured for IPv4, but this property is added into the hadoop-env.sh

export HADOOP_OPTS="-Djava.net.preferIPv4Stack=true"

I'm sure you also add a similar variable into the yarn-env.sh for YARN_OPTS, I think

However, it's not really clear from the your question when / if you've even submitted an application for anything to appear

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
  • Thank you. Follow your instructions, all Yarn services now are running on ipv4. However, Yarn UI still does not display any information about jobs have done, even it displays that those nodes are "active". Surely, after each configuration, I did some MapReduce task to see if the task appeared on Yarn UI or not. Do you have any other suggesstion? – cdt Mar 21 '18 at 05:10