Generate SPNEGO Token Failured

Question

I tried to generate the token which can be used as the HTTP header to authenticate to the HDFS WebHDFS URL and Oozie REST API URL. I referenced the url below to have the below code to generate the Negotiate token. https://www.ibm.com/support/knowledgecenter/en/SS7JFU_8.5.5/com.ibm.websphere.express.doc/ae/tsec_SPNEGO_token.html

public class TokenCreation {
  private static final String SPNEGO_OID = "1.3.6.1.5.5.2";
  private static final String KERBEROS_OID = "1.2.840.113554.1.2.2";
  public static byte[] genToken(String principal) {
    System.setProperty("javax.security.auth.useSubjectCredsOnly", "false");
    byte[] spnegoToken = new byte[0];
    try {
      Oid spnegoMechOid = new Oid(SPNEGO_OID);
      Oid krb5MechOid = new Oid(KERBEROS_OID);
      GSSCredential clientGssCreds = null;
      GSSManager manager = GSSManager.getInstance();
      GSSName gssUserName = manager.createName(principal, GSSName.NT_USER_NAME, krb5MechOid);
      clientGssCreds = manager.createCredential(gssUserName.canonicalize(krb5MechOid),
          GSSCredential.INDEFINITE_LIFETIME,
          krb5MechOid,
          GSSCredential.INITIATE_ONLY);
      clientGssCreds.add(gssUserName,
          GSSCredential.INDEFINITE_LIFETIME,
          GSSCredential.INDEFINITE_LIFETIME,
          spnegoMechOid, GSSCredential.INITIATE_ONLY);
      GSSName gssServerName = manager.createName(principal, GSSName.NT_USER_NAME);
      GSSContext clientContext = manager.createContext(gssServerName.canonicalize(spnegoMechOid),
          spnegoMechOid,
          clientGssCreds,
          GSSContext.DEFAULT_LIFETIME);
      // optional enable GSS credential delegation
      clientContext.requestCredDeleg(true);
      // create a SPNEGO token for the target server
      spnegoToken = clientContext.initSecContext(spnegoToken, 0, spnegoToken.length);
    } catch (GSSException e) {
      e.printStackTrace();
    }
    return spnegoToken;
  }

But after running the above code, I always got the below prompt:

2019-09-25 14:12:51 760 [INFO] [pool-2-thread-1] c.s.n.c.u.security.KrbUtils - after loginUserFromKeytab............AtoimcUser:HTTP/host1.exmaple.com@EXAMPLE.COM
2019-09-25 14:12:51 760 [INFO] [pool-2-thread-1] c.s.n.app.oozie.OozieAppCaller - ->>>>>>User Name is HTTP/host1.exmaple.com@EXAMPLE.COM
2019-09-25 14:12:51 760 [INFO] [pool-2-thread-1] c.s.n.app.oozie.OozieAppCaller - ->>>>>>Mode is KERBEROS
>>>KinitOptions cache name is /tmp/krb5cc_0
Kerberos username [root]: ^C^C^C
Kerberos password for root:

You can see at the end of the above output log. The "Kerberos username" is always prompt to ask for username.

Also I have tried to manually run kinit the keytab. and the above class can generate the token successfully. But manually run kinit is NOT the way I wanted.

Would you please help it? Thanks.

I did some search and found seems the above TokenCreation class can NOT generate token after running UserGroupInformation.loginUserFromKeytab? — JMS, Sep 25 '19 at 09:53
Could you explain why you are messing with low-level Kerberos credentials management, while the native Java HTTP client supports SPNego out of the box?? _(other HTTP clients e.g. from Apache may have bugs though)_ — Samson Scharfrichter, Sep 25 '19 at 19:11
Hadoop `UserGroupInformation` uses a custom lib that (partially) overrides the default Java implementation of Kerberos (inside JAAS), for a mix of good and bad reasons (both implementations are limited and brittle and hard to debug anyway) -- and when you use the UGI to "login from keytab", it always creates a **private** ticket that is not pushed to the ticket cache. Hence you can't use it from JAAS. — Samson Scharfrichter, Sep 25 '19 at 19:16
To enable the debug traces in JAAS (dumped to StdOut / StdErr) use `-Dsun.security.krb5.debug=true` and `-Djava.security.debug=gssloginconfig,configfile,configparser‌,logincontext` >> the output is NOT pretty, as you can expect from a crypto library... — Samson Scharfrichter, Sep 25 '19 at 19:18
...and also `-Dsun.security.spnego.debug=true` when using the native HTTP client — Samson Scharfrichter, Sep 25 '19 at 19:25
PS: when I mention SPNego with the "native Java HTTP client" I mean something like the code snippet in https://stackoverflow.com/questions/36287733/how-to-save-kerberos-service-ticket-using-a-windows-java-client/36303213 — Samson Scharfrichter, Sep 25 '19 at 19:33
PPS: for a JAAS config that request creating a private Kerberos ticket from a keytab (for JDBC in that case but it doesn't matter), cf. https://stackoverflow.com/questions/42477466/error-when-connect-to-impala-with-jdbc-under-kerberos-authrication/42506620 — Samson Scharfrichter, Sep 25 '19 at 19:36
Caveat - if you are using an IBM JDK, the syntax of the JAAS config is different from the "standard" syntax used by OpenJDK and Oracle. — Samson Scharfrichter, Sep 25 '19 at 19:40
@SamsonScharfrichter Thanks very much since I am new for Kerberos, and since the historical reason, I can not use "native java http client" in our code but can only by get the spnego token as the HTTP header to send out requests. I checked your example above by using UserGroupInformation's doAs method to get access hive, but do you know how can I get the token which can be used to access the kerberos's webhdfs url(http://X.X.X.X:50070/..) from ugi? — JMS, Sep 26 '19 at 06:32
I already have access to the keytab file and using UserGroupInformation.loginUserFromKeytabAndReturnUGI(userCode,keytabPath); — JMS, Sep 26 '19 at 06:35
PS: I trying to get the token from the UGI and attach it as webhdfs rest api httphttp request header like "Authorization:Negotiate " to list the remote hdfs files, not sure if this way can working. Tks so much. @SamsonScharfrichter — JMS, Sep 26 '19 at 07:21

score 1 · Answer 1 · answered Sep 26 '19 at 09:59

1

Kerberos and SPNEGO support in Java is cumbersome unfortunately.

I've created a small library to simplify some Kerberos use cases: https://github.com/bedrin/kerb4j You can use it like this to generate SPNEGO token:

SpnegoClient spnegoClient = SpnegoClient.loginWithKeyTab("svc_consumer", "/opt/myapp/consumer.keytab");
URL url = new URL("http://api.provider.acme.com/api/operation1");
SpnegoContext context = spnegoClient.createContext("http://provider.acme.com"); // Will result in HTTP/provider.acme.com SPN
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
conn.setRequestProperty("Authorization", context.createTokenAsAuthroizationHeader());

answered Sep 26 '19 at 09:59

bedrin

4,458
32
53

Sorry I am a new for kerberos, if I want to access httpfs url, e.g. http<://ip_address:50070/webhdfs/v1/user/hive/data/partition=1?&op=LISTSTATUS>, which part of URL in your code should be replaced? May I know what is the meaning of http<://provider.acme.com>? Tks. – JMS Sep 26 '19 at 10:18
Since with historical reason, I have to rely on the original Jersey client in my project, so I just want to extract the token and then add it into the request header like "Authorization:Negotiate ", not sure if it is working. – JMS Sep 26 '19 at 10:21
context.createTokenAsAuthroizationHeader() would return you the token. In order to generate it you need to provide the target host name which is registered in Kerberos server (Active Directory?) - this is the meaning of http://provider.acme.com – bedrin Sep 26 '19 at 15:07
I used your way to generate token, but found only one instance in my code can get authorization successfully, but others always return “Request is a replay 34", do you have an idea about how to fix it? Tks. – JMS Oct 07 '19 at 09:03

Generate SPNEGO Token Failured

1 Answers1