3

I am running a hadoop using the apache-hadoop binary and i have started dfs, yarn and mr deamon using the following commands:

start-dfs.sh
start-yarn.sh
mr-jobhistory-daemon.sh start historyserver

After this everything is working fine, viz, i could see the HDFS UI, resource amanger UI, Job History UI etc.

Here is the jps command output:

sh-3.2# jps
61601 Jps
sh-3.2#

But, when i am checking which processes are running using the jps command, it is showing no processes running, no name node, no datanode, no resource manager, no secondarynamenode. Even though, i could comfortably copy files from and to local to HDFS, make directory on HDFS etc.

My question is why the jps command is not showing the processes which are actually running? Do we need to invoke jps with some other argument to check the processes?

KayV
  • 12,987
  • 11
  • 98
  • 148

1 Answers1

2

Quoting from the jps documentation:

The list of JVMs produced by the jps command can be limited by the permissions granted to the principal running the command. The command only lists the JVMs for which the principle has access rights as determined by operating system-specific access control mechanisms.

In practical terms, this typically means that jps only prints information for JVM processes owned by the same user running jps. In a typical Hadoop cluster, the daemons are run under different users, e.g. user hdfs for the NameNode and user yarn for the ResourceManager. If you have sudo access and run jps as one of these users, then you can see the processes.

Chris Nauroth
  • 9,614
  • 1
  • 35
  • 39
  • my hadoop cluster was running using the KV user. However, i still tried sudo su and ran the jps, but no luck. – KayV Jun 21 '17 at 16:42
  • I have a case where the java processes are running under the same user I'm logged in as, and yet jps doesn't display them. I've also tried jps as root, and it's the same. I'm quite puzzled. – Olivier Gérardin May 13 '20 at 10:14
  • Problem was related to /tmp/hsperfdata* -> https://stackoverflow.com/questions/3805376/jps-returns-no-output-even-when-java-processes-are-running – Olivier Gérardin May 13 '20 at 10:17