I am trying the below piece of code to connect to hdfs and do some file related operation. Please note I am trying to connect a Cloudera HDFS instance from a Centos7 environment with python3.6 installed into it.
import io
from csv import reader
from krbcontext import krbcontext
from hdfs.ext.kerberos import KerberosClient
def getKerberosClient(host, port, root):
url = 'http://{host}:{port}/'.format(host=host, port=port)
client = KerberosClient(url, root=root)
return client
def main():
try:
krb = krbcontext(using_keytab=True,principal='svcid213@AMER.SOMEORG.COM',keytab_file="/nfs-hdfs/kerberos/svcid213_AMER.SOMEORG.COM.keytab")
print(F'kerberos authentication successful {krb}')
kerberos_client = getKerberosClient('deltanileuat01.amer.someorg.com', 14000, "/")
print(F'kerberos_client {kerberos_client}')
result = kerberos_client.status("/")
print(F"Output ============\n {result} \n =========================")
except Exception as e:
print(F"Detailed error is : {e} ")
if __name__ == '__main__':
main()
I am running it from an centos-7 environment with python3.6 installed [ PS: no hdfs client installed ].
Before I executed the code I have have done the pip install as below
pip3.6 install kerberos
pip3.6 install requests-kerberos
pip3.6 install hdfs
On executing I am getting as below
kerberos authentication successful <krbcontext.context.krbContext object at 0x7f1f7269c208>
kerberos_client <KerberosClient(url='http://deltanileuat01.amer.someorg.com:14000/')>
Detailed error is : Authentication failure. Check your credentials.
I am probably missing on some configuration here.
This keytab is working all good with JAVA application to connect to the hdfs. With python it is showing up this error.
Please advise that to execute the above code if
a) The HDFS client needs to be configured at the execution environment
OR
b) kinit is something that needs to be done. Please note that we are using a service id to connect to the hdfs with the keytab file.
