4

I have a spring boot oauth2 server that uses a JDBC implementation. It is configured as an authorization server with @EnableAuthorizationServer.

I'd like to scale that application horyzontally but it doesn't seem to work properly.

I can connect only if I have one instance (pods) of the server.

I use autorisation_code_client grant from another client service to get the token. So first the client service redirect the user to the oauth2 server form, then once the user is authenticated he is supposed to be redirect to the client-service with a code attached to the url, finally the client use that code to request the oauth2 server again and obtain the token.

Here the user is not redirected at all if I have several instance of the oauth2-server. With one instance it works well.

When I check the log of the two instances in real time, I can see that the authentication works on one of them. I don't have any specific error the user is just not redirected.

Is there a way to configure the oauth2-server to be stateless or other way to fix that issue ?

Here is my configuration, the AuthorizationServerConfigurerAdapter implementation.

@Configuration
public class AuthorizationServerConfiguration extends AuthorizationServerConfigurerAdapter {


    @Bean
    @ConfigurationProperties(prefix = "spring.datasource")
    public DataSource oauthDataSource() {
        return DataSourceBuilder.create().build();
    }

    @Autowired
    @Qualifier("authenticationManagerBean")
    private AuthenticationManager authenticationManager;

    @Bean
    public JdbcClientDetailsService clientDetailsSrv() {
        return new JdbcClientDetailsService(oauthDataSource());
    }

    @Bean
    public TokenStore tokenStore() {
        return new JdbcTokenStore(oauthDataSource());
    }

    @Bean
    public ApprovalStore approvalStore() {
        return new JdbcApprovalStore(oauthDataSource());
    }

    @Bean
    public AuthorizationCodeServices authorizationCodeServices() {
        return new JdbcAuthorizationCodeServices(oauthDataSource());
    }

    @Bean
    public TokenEnhancer tokenEnhancer() {

        return new CustomTokenEnhancer();
    }

    @Bean
    @Primary
    public AuthorizationServerTokenServices tokenServices() {


        DefaultTokenServices tokenServices = new DefaultTokenServices();

        tokenServices.setTokenStore(tokenStore());

        tokenServices.setTokenEnhancer(tokenEnhancer());

        return tokenServices;
    }

    @Override
    public void configure(ClientDetailsServiceConfigurer clients) throws Exception {

        clients.withClientDetails(clientDetailsSrv());
    }

    @Override
    public void configure(AuthorizationServerSecurityConfigurer oauthServer)  {

        oauthServer
                .tokenKeyAccess("permitAll()")
                .checkTokenAccess("isAuthenticated()")
                .allowFormAuthenticationForClients();

    }

    @Override
    public void configure(AuthorizationServerEndpointsConfigurer endpoints)  {
        endpoints
                .authenticationManager(authenticationManager)
                .approvalStore(approvalStore())
                //.approvalStoreDisabled()
                .authorizationCodeServices(authorizationCodeServices())
                .tokenStore(tokenStore())
                .tokenEnhancer(tokenEnhancer());
    }

}

The main class

@SpringBootApplication
@EnableResourceServer
@EnableAuthorizationServer
@EnableConfigurationProperties
@EnableFeignClients("com.oauth2.proxies")
public class AuthorizationServerApplication {


    public static void main(String[] args) {

        SpringApplication.run(AuthorizationServerApplication.class, args);

    }

}

The Web Security Configuration

@Configuration
@Order(1)
public class WebSecurityConfiguration extends WebSecurityConfigurerAdapter {


    @Bean
    @Override
    public UserDetailsService userDetailsServiceBean() throws Exception {
        return new JdbcUserDetails();
    }

    @Override
    @Bean
    public AuthenticationManager authenticationManagerBean() throws Exception {
        return super.authenticationManagerBean();
    }

    @Override
    protected void configure(HttpSecurity http) throws Exception { // @formatter:off

        http.requestMatchers()
                .antMatchers("/",
                        "/login",
                        "/login.do",
                        "/registration",
                        "/registration/confirm/**",
                        "/registration/resendToken",
                        "/password/forgot",
                        "/password/change",
                        "/password/change/**",
                        "/oauth/authorize**")
                .and()
                .authorizeRequests()//autorise les requetes
                .antMatchers(
                        "/",
                        "/login",
                        "/login.do",
                        "/registration",
                        "/registration/confirm/**",
                        "/registration/resendToken",
                        "/password/forgot",
                        "/password/change",
                        "/password/change/**")
                .permitAll()
                .and()
                .requiresChannel()
                .anyRequest()
                .requiresSecure()
                .and()
                .authorizeRequests()
                .anyRequest()
                .authenticated()
                .and()
                .formLogin()
                .loginPage("/login")
                .loginProcessingUrl("/login.do")
                .usernameParameter("username")
                .passwordParameter("password")
                .and()
                .userDetailsService(userDetailsServiceBean());


    } // @formatter:on


    @Override
    protected void configure(AuthenticationManagerBuilder auth) throws Exception {
        auth.userDetailsService(userDetailsServiceBean()).passwordEncoder(passwordEncoder());
    }


}

Client side the WebSecurityConfigurerAdapter

@EnableOAuth2Sso
@Configuration
public class UiSecurityConfig extends WebSecurityConfigurerAdapter {

    @Override
    public void configure(HttpSecurity http) throws Exception {

        http.antMatcher("/**")
                .authorizeRequests()
                .antMatchers(
                        "/",
                        "/index.html",
                        "/login**",
                        "/logout**",
                        //resources
                        "/assets/**",
                        "/static/**",
                        "/*.ico",
                        "/*.js",
                        "/*.json").permitAll()
                .anyRequest()
                .authenticated()
                .and()
                .csrf().csrfTokenRepository(csrfTokenRepository())
                .and()
                .addFilterAfter(csrfHeaderFilter(), SessionManagementFilter.class);
    }

}

the oauth2 configuration properties

oauth2-server is the service name (load balancer) on kubernetes and also the server path that is why it appears twice.

security:
    oauth2:
        client:
            clientId: **********
            clientSecret: *******
            accessTokenUri: https://oauth2-server/oauth2-server/oauth/token
            userAuthorizationUri: https://oauth2.mydomain.com/oauth2-server/oauth/authorize
        resource:
            userInfoUri: https://oauth2-server/oauth2-server/me

Here an important detail, the value of userAuthorizationUri is the address to access the oauth2-server from the outside of the k8s cluster. The client-service send back that address into the response with a 302 http code if the user is not connected and tries to access to the /login path of the client-service. then the user is redirected to the /login path of the oauth2-server.
https://oauth2.mydomain.com target an Nginx Ingress controller that handle the redirection to the load balancer service.

kaizokun
  • 926
  • 3
  • 9
  • 31

2 Answers2

5

Here is a solution to this problem. It's not a Spring issue at all but a bad configuration of the Nginx Ingress controller.

The authentication process is done in several stages :

1 - the user clic on a login button that target the /login path of the client-server

2 - the client-server, if the user is not authenticated yet, send a response to the browser with a 302 http code to redirect the user to the oauth2-server, the value of the redirection is composed with the value of the security.oauth2.client.userAuthorizationUri property and the redirection url that will be used by the browser to allow the client-server to get the Token once the user is authenticated. That url look like this :

h*tps://oauth2.mydomain.com/oauth2-server/oauth/authorize?client_id=autorisation_code_client&redirect_uri=h*tps://www.mydomain.com/login&response_type=code&state=bSWtGx

3 - the user is redirected to the previous url

4 - the oauth2-server send a 302 http code to the browser with the login url of the oauth2-server, h*tps://oauth2.mydomain.com/oauth2-server/login

5 - the user submit his credentials and the token is created if they are correct.

6 - the user is redirected to the same address as at the step two, and the oauth-server add informations to the redirect_uri value

7 - the user is redirected to the client-server. The redirection part of the response look like this :

location: h*tps://www.mydomain.com/login?code=gnpZ0r&state=bSWtGx

8 - the client-server contact the oauth2-server and obtain the token from the code and the state that authenticates it. It doesn't matter if the instance of the oauth2 server is different than the one used by the user to authenticate himself. Here the client-server use the value of security.oauth2.client.accessTokenUri to get the token, this is the internal load balancing service address that targets the oauth2 server pods, so it doesn't pass through any Ingress controller.

So at the steps 3 to 6 the user must communicate with the same instance of the oauth2-server throught the Ingress controller in front of the load balancer service.

Its is possible by configuring the Nginx Ingress controller with a few annotations :

"annotations": {
  ...
  "nginx.ingress.kubernetes.io/affinity": "cookie",
  "nginx.ingress.kubernetes.io/session-cookie-expires": "172800",
  "nginx.ingress.kubernetes.io/session-cookie-max-age": "172800",
  "nginx.ingress.kubernetes.io/session-cookie-name": "route"
}

That way we ensure that the user will be redirected to the same pods/instance of the oauth2-server during the authentication process as long he's identified with the same cookie.

The affinity session mecanism is a great way to scale the authentication server and also the client-server. Once the user is authenticated he will always use the same instance of the client and keep his session informations.

Thanks to Christian Altamirano Ayala for his help.

kaizokun
  • 926
  • 3
  • 9
  • 31
  • Good to see that you found out the issue – Jonas Oct 24 '19 at 15:54
  • 2
    Thanks, it was more easy to fix than I thought finally. But I needed a better comprehension of the whole process. I Hope it will help someone else. – kaizokun Oct 24 '19 at 16:04
  • thank you for this comprehensive explanation. I was able to resolve my issue using this approach. I had to use different annotations as using voyager/haproxy (`ingress.appscode.com/affinity: 'cookie'`) but it worked all the same. If it wasn't for this answer, I would have spent a long time poking around Spring code, cursing them completely without reason. I don't really understand why the authentication works against a different pod once the initial round trip is completed ( I tested by logging in using pod A, then switching to pod B) but it does – James Render Jan 28 '20 at 15:48
  • for anyone using voyager here's a quick link to the [sticky session](https://appscode.com/products/voyager/8.0.1/guides/ingress/http/sticky-session/) guide - make sure you get the right version as they've changed the annotation name several times (by logging onto a voyager pod and `voyager version`) – James Render Jan 28 '20 at 15:57
2

By default an in-memory TokenStore is used.

The default InMemoryTokenStore is perfectly fine for a single server

If you want multiple pods, you probably should go for JdbcTokenStore

The JdbcTokenStore is the JDBC version of the same thing, which stores token data in a relational database. Use the JDBC version if you can share a database between servers, either scaled up instances of the same server if there is only one, or the Authorization and Resources Servers if there are multiple components. To use the JdbcTokenStore you need "spring-jdbc" on the classpath.

Source Spring Security: OAuth 2 Developers Guide

Jonas
  • 121,568
  • 97
  • 310
  • 388
  • Actually the configuration already use a JdbcTokenStore. I don't understand why, even if I have two instance of the server, the authentication works and the redirection doesn't happen. Because the instance that perform the authentication should also perform the redirection, but nothing happen. If the client receive the code and request another instance of the oauth2 server configured with a in-memory tokenStore, it makes sens, but here it 's supposed to use JDBC. Perhaps the client need the same session to get the token, that is not shared, but here I don't think the client made that request. – kaizokun Oct 23 '19 at 18:28