2

I have setup a hive environment with Kerberos security enabled on a Linux server (Red Hat). And I need to connect from a remote windows machine to hive using JDBC.

So, I have hiveserver2 running in the linux machine, and I have done "kinit".

Now I try to connect from a java program on the windows side with a test program like this,

Class.forName("org.apache.hive.jdbc.HiveDriver");
String url = "jdbc:hive2://<host>:10000/default;principal=hive/_HOST@<YOUR-REALM.COM>"
Connection con = DriverManager.getConnection(url);

And I got the following error,

Exception due to: Could not open client transport with JDBC Uri:
 jdbc:hive2://<host>:10000/;principal=hive/_HOST@YOUR-REALM.COM>: 
GSS initiate failed

What am I doing here wrong ? I checked many forums, but couldn't get a proper solution. Any answer will be appreciated.

Thanks

abisheksampath
  • 376
  • 8
  • 23
  • OK, so you have now two different solutions: install (properly) the MIT Kerberos Windows client (with your custom Kerberos config), and hope that JAAS finds the default ticket cache automatically without any additional config; or don't install anything and simply attach a (proper) JAAS config file (plus your custom Kerberos config). Choose one. – Samson Scharfrichter Sep 08 '16 at 08:08
  • Note that the "MIT Kerberos client" solution requires a `kinit` every day, which complicates the scheduling of batch executions. – Samson Scharfrichter Sep 08 '16 at 08:11
  • Any luck so far? Do I need to install Kerberos on window? – Hoang Minh Quang FX15045 May 11 '23 at 08:20

3 Answers3

2

If you were running your code in Linux, I would simply point to that post -- i.e. you must use System properties to define Kerberos and JAAS configuration, from conf files with specific formats.
And you have to switch the debug trace flags to understand subtile configuration issue (i.e. different flavors/versions of JVMs may have different syntax requirements, which are not documented, it's a trial-and-error process).

But on Windows there are additional problems:

  1. the Apache Hive JDBC driver has some dependencies on Hadoop JARs, especially when Kerberos is involved (see that post for details)
  2. these Hadoop JARs require "native libraries" -- i.e. a Windows port of Hadoop (which you have to compile yourself!! or download from an insecure source on the web!!) -- plus System properties hadoop.home.dir and java.library.path pointing to the Hadoop home dir and its bin sub-dir respectively

On the top of that, the Apache Hive driver has compatibility issues -- whenever there are changes in the wire protocol, newer clients cannot connect to older servers.

So I strongly advise you to use the Cloudera JDBC driver for Hive for your Windows clients. The Cloudera site just asks your e-mail.
After that you have a 80+ pages PDF manual to read, the JARs to add to your CLASSPATH, and your JDBC URL to adapt according to the manual.
Side note: the Cloudera driver is a proper JDBC-4.x compliant driver, no need for that legacy Class.forName()...

Community
  • 1
  • 1
Samson Scharfrichter
  • 8,884
  • 1
  • 17
  • 36
  • If you have to debug Kerberos error messages, look at that GitBook https://steveloughran.gitbooks.io/kerberos_and_hadoop/content/sections/errors.html under sections "errors" and "terrors". The whole book is a must-read if you have to build systems that use a Kerberized Hadoop cluster. – Samson Scharfrichter Sep 07 '16 at 08:52
  • Thanks for your reply. I tried it with the cloudera hive jdbc jar, and also I had already referred to other necessary libraries in the classpath. so I don't think that is the issue. I would like to point out that, the windows client doesn't have any sort of Kerberos setup (kerberos setup only in Linux machine) and also, if not for Kerberos, im able to connect to the hiveserver2 instance. So, this might be vague, but do I need Kerberos in windows too ? – abisheksampath Sep 07 '16 at 14:17
  • *"the windows client doesn't have any sort of Kerberos setup"* -- that's the point addressed in the first link in my answer i.e. **you must** create a valid `krb5.conf` and a valid JAAS conf file, then point to these files with Java system props. – Samson Scharfrichter Sep 07 '16 at 14:21
0

The key for us when we ran into the problem, was as follows:

On your server there are certain kerberos principals listed that are allowed to operate on the data.

When we tried to run a query via JDBC, we didn't do the proper kinit on the client side.

In this case the solution is obvious:

On the windows client: do a kinit with the proper account before connecting

Community
  • 1
  • 1
Dennis Jaheruddin
  • 21,208
  • 8
  • 66
  • 122
0
String url = "jdbc:hive2://<host>:10000/default;principal=hive/_HOST@<YOUR-REALM.COM>"

You should replace <YOUR-REALM.COM> with your real REALM.

Jonny Young
  • 31
  • 6