3

I've followed the following tutorial CDH Hadoop Kerberos, NameNode and DataNode are able to start properly and I'm able to see all the DataNode listed on the WebUI (0.0.0.0:50070). But I'm unable to access the Hadoop CLI. I've followed this tutorial Certain Java versions cannot read credentials cache, still I'm unable to use the Hadoop CLI.

[root@local9 hduser]# hadoop fs -ls /
20/11/03 12:24:32 WARN security.UserGroupInformation: PriviledgedActionException as:root (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
20/11/03 12:24:32 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
20/11/03 12:24:32 WARN security.UserGroupInformation: PriviledgedActionException as:root (auth:KERBEROS) cause:java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
ls: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "local9/192.168.2.9"; destination host is: "local9":8020;
[root@local9 hduser]# klist
Ticket cache: KEYRING:persistent:0:krb_ccache_hVEAjWz
Default principal: hdfs/local9@FBSPL.COM

Valid starting       Expires              Service principal
11/03/2020 12:22:42  11/04/2020 12:22:42  krbtgt/FBSPL.COM@FBSPL.COM
        renew until 11/10/2020 12:22:12
[root@local9 hduser]# kinit -R
[root@local9 hduser]# klist
Ticket cache: KEYRING:persistent:0:krb_ccache_hVEAjWz
Default principal: hdfs/local9@FBSPL.COM

Valid starting       Expires              Service principal
11/03/2020 12:24:50  11/04/2020 12:24:50  krbtgt/FBSPL.COM@FBSPL.COM
        renew until 11/10/2020 12:22:12
[root@local9 hduser]# hadoop fs -ls /
20/11/03 12:25:04 WARN security.UserGroupInformation: PriviledgedActionException as:root (auth:KERBEROS) cause:javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
20/11/03 12:25:04 WARN ipc.Client: Exception encountered while connecting to the server : javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
20/11/03 12:25:04 WARN security.UserGroupInformation: PriviledgedActionException as:root (auth:KERBEROS) cause:java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]
ls: Failed on local exception: java.io.IOException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)]; Host Details : local host is: "local9/192.168.2.9"; destination host is: "local9":8020;

Any Help would be greatly appreciated.

Saif Ahmad
  • 1,118
  • 1
  • 8
  • 24
  • Your first link says _"For Hadoop, the principals should be of the format username/**fully.qualified.domain.name**@YOUR-REALM.COM"_. `local9` is not a fully qualified domain name. – mazaneicha Nov 06 '20 at 15:14
  • local9 is the fully qualified domain name. *hostname -f* gives local9 as the output. – Saif Ahmad Nov 07 '20 at 16:06
  • No its not, https://www.lifewire.com/what-does-fqdn-mean-2625883. – mazaneicha Nov 07 '20 at 16:53
  • I've set the hostname to local9 using hostnamectl -set-hostname and /etc/hostname to local9. Hence my fqdn is local9 – Saif Ahmad Nov 08 '20 at 17:10

1 Answers1

2

I figured out the issue. It's a cache credential bug in Redhat: Red Hat Bugzilla – Bug 1029110 Then I found this document on Kerberos on Cloudera: Manage krb5.conf

Finally the solution was to comment out this line from /etc/krb5.conf

default_ccache_name = KEYRING:persistent:%{uid}

I was able to access the Hadoop CLI after commenting this line.

Saif Ahmad
  • 1,118
  • 1
  • 8
  • 24