How to configure roles based on groups from HadoopGroupProvider in Zeppelin, using Knox to provide SAML-based SSO?

Question

I am trying to implement Role Base Access Controls on Zeppelin, using Knox to authenticate against an external IdP, and to perform group lookups from an LDAP instance once a user has been successfully authenticated.

I am currently able to login to Zeppelin, and the HadoopGroupProvider is looking up the user's groups as expected, but the authenticated user is not mapped to any roles, and is therefore unable to create Notebooks, or use any interpreters.

My configuration for Knox is shown below:

<?xml version="1.0" encoding="utf-8"?>
<topology>
    <gateway>
      <provider>
        <role>federation</role>
        <name>pac4j</name>
        <enabled>true</enabled>
        <param>
          <name>pac4j.callbackUrl</name>
          <value>https://knox.example.com/gateway/knoxsso/api/v1/websso</value>
        </param>
        <param>
          <name>clientName</name>
          <value>SAML2Client</value>
        </param>
        <param>
          <name>saml.keystorePath</name>
          <value>/opt/knox-1.3.0/data/security/keystores/gateway.jks</value>
        </param>
        <param>
          <name>saml.keystorePassword</name>
          <value>password</value>
        </param>
        <param>
          <name>saml.privateKeyPassword</name>
          <value>password</value>
        </param>
        <param>
          <name>saml.identityProviderMetadataPath</name>
          <value>/etc/sso/idp.xml</value>
        </param>
        <param>
          <name>saml.maximumAuthenticationLifetime</name>
          <value>100000</value>
        </param>
        <param>
          <name>saml.serviceProviderEntityId</name>
          <value>https://knox.example.com/gateway/knoxsso/api/v1/websso?pac4jCallback=true&amp;client_name=SAML2Client</value>
        </param>
        <param>
          <name>saml.serviceProviderMetadataPath</name>
          <value>/etc/sso/sp.xml</value>
        </param>
        <param>
          <name>pac4j.id_attribute</name>
          <value>username</value>
        </param>
      </provider>
      <provider>
        <role>identity-assertion</role>
        <name>HadoopGroupProvider</name>
        <enabled>true</enabled>
        <param>
            <name>hadoop.security.group.mapping</name>
            <value>org.apache.hadoop.security.LdapGroupsMapping</value>
        </param>
        <param>
            <name>hadoop.security.group.mapping.ldap.bind.user</name>
            <value>cn=loginuser,ou=example,ou=example,dc=example,dc=example,dc=example,dc=com</value>
        </param>
        <param>
            <name>hadoop.security.group.mapping.ldap.bind.password</name>
            <value>password</value>
        </param>
        <param>
            <name>hadoop.security.group.mapping.ldap.url</name>
            <value>ldap://example.ldap.com:389</value>
        </param>
        <param>
            <name>hadoop.security.group.mapping.ldap.base</name>
            <value>ou=example,dc=example,dc=example,dc=example,dc=com</value>
        </param>
        <param>
            <name>hadoop.security.group.mapping.ldap.search.filter.user</name>
            <value>(&amp;(objectClass=user)(|(sAMAccountName={0})(mailNickname={0})))</value>
        </param>
        <param>
            <name>hadoop.security.group.mapping.ldap.search.filter.group</name>
            <value>(&amp;(cn=group*)(objectclass=Group))</value>
        </param>
        <param>
            <name>hadoop.security.group.mapping.ldap.search.attr.member</name>
            <value>member</value>
        </param>
        <param>
            <name>hadoop.security.group.mapping.ldap.search.attr.group.name</name>
            <value>cn</value>
        </param>
      </provider>
    </gateway>
    <service>
        <role>KNOXSSO</role>
        <param>
           <name>knoxsso.cookie.secure.only</name>
           <value>true</value>
        </param>
        <param>
          <name>knoxsso.token.ttl</name>
          <value>100000</value>
        </param>
        <param>
          <name>knoxsso.redirect.whitelist.regex</name>
          <value>.*</value>
        </param>
        <param>
            <name>knoxsso.token.ttl</name>
            <value>-1</value>
        </param>
    </service>
</topology>

This is my shiro.ini configuration for Zeppelin:

[main]
knoxJwtRealm = org.apache.zeppelin.realm.jwt.KnoxJwtRealm
knoxJwtRealm.providerUrl = https://knox.example.com/
knoxJwtRealm.login = gateway/knoxsso/api/v1/websso
knoxJwtRealm.publicKeyPath = /etc/pki/tls/certs/knox.example.com.pem
knoxJwtRealm.logoutAPI = false
knoxJwtRealm.logout = gateway/knoxsso/api/v1/webssout
knoxJwtRealm.cookieName = hadoop-jwt
knoxJwtRealm.redirectParam = originalUrl
knoxJwtRealm.groupPrincipalMapping = group.principal.mapping
knoxJwtRealm.principalMapping = principal.mapping
authc = org.apache.zeppelin.realm.jwt.KnoxAuthenticationFilter

securityManager.realms = $knoxJwtRealm

sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

cookie = org.apache.shiro.web.servlet.SimpleCookie
cookie.name = JSESSIONID
cookie.httpOnly = true
sessionManager.sessionIdCookie = $cookie

securityManager.sessionManager = $sessionManager
securityManager.sessionManager.globalSessionTimeout = 86400000
shiro.loginUrl = /api/login

[roles]
admin_role = *
user_role = *

[urls]
/api/version = anon
/** = authc

I am confident that the HadoopGroupProvider is connecting to my LDAP instance and successfully looking up my groups, due to the gateway-audit.log:

19/10/07 15:33:00 ||6348f279-0ed2-445b-8a73-b76a8fcb985a|audit|1.2.3.4|KNOXSSO|USER1|||identity-mapping|principal|USER1|success|Groups: [Group1, Group2, Group3]

My questions are:

How do I map these groups to roles in Zeppelin?

Is there an equivalent to the org.apache.zeppelin.realm.LdapRealm's rolesByGroup configuration for the KnoxJwtRealm?

Any help is greatly appreciated, thanks in advance!

I am facing the similar issue. Did you able to find a way through? — Jithesh Gopinathan, Nov 25 '20 at 07:21

score 1 · Answer 1 · answered Feb 21 '22 at 18:45

You need to install hadoop binaries and configure Hadoop Group Mapping. And get Zeppelin rely on this configuration by providing environment variables in zeppelin-env.sh:

USE_HADOOP=True
HADOOP_CONF_DIR=<PATH_TO_HADOOP_CONFIGURATION_FILES>

You either need to add $HADOOP_HOME/bin to your OS $PATH environment variable. So Zeppelin could run hadoop command to map users and groups.

Setup specific group access permissions by writing URL-based rules under url section, such as:

[urls]

/api/configurations/** = authc, roles[<YOUR_LDAP_GROUP>]

More info:

Hadoop integration

Zeppelin KnoxSSO

score 0 · Answer 2 · answered Oct 16 '19 at 00:56

The group lookup in the HadoopGroupProvider within the Knox topology looks correct and you would be able to use those groups for service level authorization for protecting access to the services being proxied by that topology.

However, those groups do not get propagated to the backend service from the gateway. Zeppelin needs to be configured to do its own group lookup and role mapping. I'm not familiar with the Zeppelin config for that unfortunately.

The following may prove helpful though: https://zeppelin.apache.org/docs/0.8.0/setup/security/shiro_authentication.html#groups-and-permissions-optional

How to configure roles based on groups from HadoopGroupProvider in Zeppelin, using Knox to provide SAML-based SSO?

2 Answers2