0

I have been struggling with how to pass requisite parameters to snakebite utility for it to be able to access a kerberized cluster. I have tried setting the necessary conf dir in the /usr/local/etc/hadoop path, as well as initialising and getting a ticket using kinit.

Any help or working example in this regards would be greatly appreciated.

NOTE: I have tested that the environment setup is proper by using 'hadoop' CLI to access the cluster from the same machine.

Saurabh Mishra
  • 1,703
  • 3
  • 17
  • 27

2 Answers2

2

I recommend you use autoconfigure class for this purpose:

from snakebite.client import AutoConfigClient
client = AutoConfigClient()

It supports HA and is auto configured through the HADOOP_HOME environment variable.

Please make sure that you also have a valid keytab

The klist output for the environment looks as follows (there is only 1 entry in there)

Credentials cache: API:1350E541-0CCD-46B9-938E-2C1C8E03A909
    Principal: saurabh@HADOOP_DEV.MY-COMPANY.COM

  Issued                Expires               Principal
May 31 16:45:30 2017  Jun  1 02:45:30 2017  krbtgt/HADOOP_DEV.MY-COMPANY.COM@HADOOP_DEV.MY-COMPANY.COM

This allows me to successfully list directories on the HDFS

hadoop fs -ls /user/saurabh
Saurabh Mishra
  • 1,703
  • 3
  • 17
  • 27
jedijs
  • 563
  • 5
  • 14
1

You must kinit with proper keytab is enough. It will take automatically by the principal name to fetch results.

Chitrapandi
  • 25
  • 7