HttpSession null after replacing AuthorizationRequest

Question

Complete code and instructions to quickly reproduce the problem are given below.

---

THE PROBLEM:

---

The HttpSession becomes null after a custom implementation of DefaultOAuth2RequestFactory replaces the current AuthorizationRequest with a saved AuthorizationRequest. This causes failure of the subsequent request to /oauth/token because the CsrfFilter in the Spring Security filter chain preceding the /oauth/token endpoint is not able to find a session Csrf token in the null session to compare with the request's Csrf token.

---

CONTROL FLOW DURING THE ERROR:

---

The following flowchart illustrates where Step 14 and Step 15 somehow null-ify the HttpSession. (Or possibly mismatch a JSESSIONID.) A SYSO at the start of CustomOAuth2RequestFactory.java in Step 14 shows that there is indeed an HttpSession that does in fact contain the correct CsrfToken. Yet, somehow, the HttpSession has become null by the time Step 15 triggers a call from the client at the localhost:8080/login url back to the localhost:9999/oauth/token endpoint.

Breakpoints were added to every line of the HttpSessionSecurityContextRepository mentioned in the debug logs below. (It is located in the Maven Dependencies folder of the authserver eclipse project.) These breakpoints confirmed that the HttpSession is null when the final request to /oauth/token is made in the flowchart below. (Bottom-left of flowchart.) The null HttpSession might be due to the JSESSIONID that remains in the browser becoming out of date after the custom DefaultOAuth2RequestFactory code runs.

How can this problem be fixed, so that the same HttpSession remains during the final call to the /oauth/token endpoint, after the end of Step 15 in the flowchart?

---

RELEVANT CODE AND LOGS:

---

The complete code of CustomOAuth2RequestFactory.java can be viewed at a file sharing site by clicking on this link. We can guess that the null session is due to either 1.) the JSESSIONID not being updated in the browser by the code in the CustomOAuth2RequestFactory, or 2.) the HttpSession actually being null-ified.

The Spring Boot debug logs for the call to /oauth/token after Step 15 clearly state that there is no HttpSession by that point, and can be read as follows:

2016-05-30 15:33:42.630 DEBUG 13897 --- [io-9999-exec-10] o.s.security.web.FilterChainProxy        : /oauth/token at position 1 of 12 in additional filter chain; firing Filter: 'WebAsyncManagerIntegrationFilter'
2016-05-30 15:33:42.631 DEBUG 13897 --- [io-9999-exec-10] o.s.security.web.FilterChainProxy        : /oauth/token at position 2 of 12 in additional filter chain; firing Filter: 'SecurityContextPersistenceFilter'
2016-05-30 15:33:42.631 DEBUG 13897 --- [io-9999-exec-10] w.c.HttpSessionSecurityContextRepository : No HttpSession currently exists
2016-05-30 15:33:42.631 DEBUG 13897 --- [io-9999-exec-10] w.c.HttpSessionSecurityContextRepository : No SecurityContext was available from the HttpSession: null. A new one will be created.
2016-05-30 15:33:42.631 DEBUG 13897 --- [io-9999-exec-10] o.s.security.web.FilterChainProxy        : /oauth/token at position 3 of 12 in additional filter chain; firing Filter: 'HeaderWriterFilter'
2016-05-30 15:33:42.631 DEBUG 13897 --- [io-9999-exec-10] o.s.s.w.header.writers.HstsHeaderWriter  : Not injecting HSTS header since it did not match the requestMatcher org.springframework.security.web.header.writers.HstsHeaderWriter$SecureRequestMatcher@2fe29f4b
2016-05-30 15:33:42.631 DEBUG 13897 --- [io-9999-exec-10] o.s.security.web.FilterChainProxy        : /oauth/token at position 4 of 12 in additional filter chain; firing Filter: 'CsrfFilter'
2016-05-30 15:33:42.644 DEBUG 13897 --- [io-9999-exec-10] o.s.security.web.csrf.CsrfFilter         : Invalid CSRF token found for http://localhost:9999/uaa/oauth/token
2016-05-30 15:33:42.644 DEBUG 13897 --- [io-9999-exec-10] w.c.HttpSessionSecurityContextRepository : SecurityContext is empty or contents are anonymous - context will not be stored in HttpSession.
2016-05-30 15:33:42.644 DEBUG 13897 --- [io-9999-exec-10] s.s.w.c.SecurityContextPersistenceFilter : SecurityContextHolder now cleared, as request processing completed

---

RE-CREATING THE PROBLEM ON YOUR COMPUTER:

---

You can recreate the problem on any computer in only a few minutes by following these simple steps:

1.) Download the zipped version of the app from a file sharing site by clicking on this link.

2.) Unzip the app by typing: tar -zxvf oauth2.tar(4).gz

3.) Launch the authserver app by navigating to oauth2/authserver and then typing mvn spring-boot:run.

4.) Launch the resource app by navigating to oauth2/resource and then typing mvn spring-boot:run

5.) Launch the ui app by navigating to oauth2/ui and then typing mvn spring-boot:run

6.) Open a web browser and navigate to http : // localhost : 8080

7.) Click Login and then enter Frodo as the user and MyRing as the password, and click to submit.

8.) Enter 5309 as the Pin Code and click submit. This will trigger the error shown above.

The Spring Boot debug logs will show A LOT of SYSO, which gives the values of variables such as XSRF-TOKEN and HttpSession at each step shown in the flowchart. The SYSO helps segment the debug logs so that they are easier to interpret. And all the SYSO is done by one class called by the other classes, so you can manipulate the SYSO-generating class to change reporting everywhere in the control flow. The name of the SYSO-generating class is TestHTTP, and its source code can be found in the same demo package.

---

USE THE DEBUGGER:

---

1.) Select the terminal window that is running the authserver app and type Ctrl-C to stop the authserver app.

2.) Import the three apps (authserver, resource, and ui) into eclipse as existing maven projects.

3.) In the authserver app's eclipse Project Explorer, click to expand the Maven Dependencies folder, then scroll down within it to click to expand the Spring-Security-web... jar as shown circled in orange in the image below. Then scroll to find and expand the org.springframework.security.web.context package. Then double click to open the HttpSessionSecurityContextRepository class highlighted in blue in the screen shot below. Add breakpoints to every line in this class. You may want to do the same to the SecurityContextPersistenceFilter class in the same package. These breakpoints will enable you to see the value of the HttpSession, which currently becomesnull before the end of the control flow, but needs to have a valid value that can be mapped to an XSRF-TOKEN in order to resolve this OP.

4.) In the app's demo package, add breakpoints inside the CustomOAuth2RequestFactory.java. Then Debug As... Spring Boot App to start the debugger.

5.) Then repeat steps 6 through 8 above. You may want to clear the browser's cache before each new attempt. And you may want the Network tab of the browser's developer tools open.

Answer 1

The session is not null in your authserver app at the time of the final call to localhost :9999/uaa/oauth/token. Not only is there a session, but the JSESSIONID and the csrf token of the valid session match values present in the control flow between the point where the user submits the correct pin and the point where the failed request to /oauth/token is made.

The problem is that there are two JSESSIONID values, and the wrong of the two values is selected to enter the call to /oauth/token. Therefore, the solution should come from modifying the filters to delete the bad JSESSIONID so that the correct value can be sent.

The following will summarize:

---

HttpSessionListener identified the valid JSESSIONID

---

To isolate the problem, I created an implementation of HttpSessionListener and then called it from a custom implementation of HttpLListener, as follows:

public class HttpSessionCollector implements HttpSessionListener, ServletContextListener {

    private static final Set<HttpSession> sessions = ConcurrentHashMap.newKeySet();

    public void sessionCreated(HttpSessionEvent event) {
        sessions.add(event.getSession());
    }

    public void sessionDestroyed(HttpSessionEvent event) {
        sessions.remove(event.getSession());
    }

    public static Set<HttpSession> getSessions() {
        return sessions;
    }

    public void contextCreated(ServletContextEvent event) {
        event.getServletContext().setAttribute("HttpSessionCollector.instance", this);
    }

    public static HttpSessionCollector getCurrentInstance(ServletContext context) {
        return (HttpSessionCollector) context.getAttribute("HttpSessionCollector.instance");
    }

    @Override
    public void contextDestroyed(ServletContextEvent arg0) {
    }

    @Override
    public void contextInitialized(ServletContextEvent arg0) {
    }

}

I then called the above HttpSessionListener in a custom implementation of OncePerRequestFilter, which I inserted into your authserver app's Spring Security Filter Chain to provide diagnostic information, as follows:

@Component
public class DiagnoseSessionFilter extends OncePerRequestFilter implements ServletContextAware {

    @Override
    protected void doFilterInternal(HttpServletRequest req, HttpServletResponse res, FilterChain fc) throws ServletException, IOException {

    System.out.println("...........///////////// START OF DiagnoseSessionFilter.doFilterInternal() ///////////...........");
    //start of request stuff
    System.out.println("\\\\\\\\\\ REQUEST ATTRIBUTES ARE: ");
    if(req.getAttribute("_csrf")!=null){
        System.out.println("_csrf is: " + req.getAttribute("_csrf").toString());
    }
    if(req.getAttribute("org.springframework.security.web.csrf.CsrfToken")!=null){
        CsrfToken ucsrf = (CsrfToken) req.getAttribute("org.springframework.security.web.csrf.CsrfToken");
        System.out.println("ucsrf.getToken() is: " + ucsrf.getToken());
    }
    String reqXSRF = req.getHeader("XSRF-TOKEN");
    System.out.println("request XSRF-TOKEN header is: " + reqXSRF);
    String reqCookie = req.getHeader("Cookie");
    System.out.println("request Cookie header is: " + reqCookie);
    String reqSetCookie = req.getHeader("Set-Cookie");
    System.out.println("request Set-Cookie header is: " + reqSetCookie);
    String reqReferrer = req.getHeader("referrer");
    System.out.println("request referrer header is: " + reqReferrer);
    HttpSession rsess = req.getSession(false);
    System.out.println("request.getSession(false) is: " + rsess);
    if(rsess!=null){
        String sessid = rsess.getId();
        System.out.println("session.getId() is: "+sessid);
    }
    System.out.println("/////////// END OF REQUEST ATTRIBUTES ");

    //end of request stuff
    ServletContext servletContext = req.getServletContext();
    System.out.println("\\\\\\\\\\ START OF SESSION COLLECTOR STUFF ");

    HttpSessionCollector collector = HttpSessionCollector.getCurrentInstance(servletContext);
    Set<HttpSession> sessions = collector.getSessions();

    System.out.println("sessions.size() is: " + sessions.size());
    for(HttpSession sess : sessions){
        System.out.println("sess is: " + sess);
        System.out.println("sess.getId() is: " + sess.getId());
        CsrfToken sessCsrf = (CsrfToken) sess.getAttribute("org.springframework.security.web.csrf.HttpSessionCsrfTokenRepository.CSRF_TOKEN");
        System.out.println("csrf is: " + sessCsrf);
        if(sessCsrf!=null){
            if(sessCsrf.getToken()!=null){
                System.out.println("sessCsrf.getToken() is: " + sessCsrf.getToken());
            } else { System.out.println("sessCsrf.getToken() is: null "); }
        } else { System.out.println("sessCsrf is: null "); }

        System.out.println("sess.getAttribute(SPRING_SECURITY_SAVED_REQUEST) is: " + sess.getAttribute("SPRING_SECURITY_SAVED_REQUEST") );
        if(sess.getAttribute("SPRING_SECURITY_SAVED_REQUEST") instanceof DefaultSavedRequest){
            System.out.println("_____ START PRINTING SAVED REQUEST");
            DefaultSavedRequest savedReq = (DefaultSavedRequest) sess.getAttribute("SPRING_SECURITY_SAVED_REQUEST");
            List<Cookie> savedCookies = savedReq.getCookies();
            for(Cookie cook : savedCookies){
                String name = cook.getName();String value = cook.getValue();
                System.out.println("cookie name, value are: " + name + " , " + value);
            }
            Collection<String> savedHeaderNames = savedReq.getHeaderNames();
            for(String headerName : savedHeaderNames){
                System.out.println("headerName is: " + headerName);
            }
            List<Locale> savedLocales = savedReq.getLocales();
            for(Locale loc : savedLocales){
                System.out.println("loc.getLanguage() is: " + loc.getLanguage());
            }
            String savedMethod = savedReq.getMethod();
            System.out.println("savedMethod is: " + savedMethod);
            Map<String, String[]> savedParamMap = savedReq.getParameterMap();
            Iterator<Entry<String, String[]>> it = savedParamMap.entrySet().iterator();
            while (it.hasNext()) {
                Entry<String, String[]> pair = it.next();
                System.out.println("savedParamMap: " + pair.getKey() + " = " + pair.getValue());
                it.remove(); // avoids a ConcurrentModificationException
            }
            Collection<String> savedParamNames = savedReq.getParameterNames();
            for(String savedParamName : savedParamNames){
                System.out.println("savedParamName: " + savedParamNames);
            }
            System.out.println("_____ DONE PRINTING SAVED REQUEST");

        }

//      System.out.println("sess.getAttribute(SPRING_SECURITY_CONTEXT) is: " + sess.getAttribute("SPRING_SECURITY_CONTEXT") );
        if(sess.getAttribute("SPRING_SECURITY_CONTEXT") instanceof SecurityContextImpl){
            SecurityContext ctxt = (SecurityContext) sess.getAttribute("SPRING_SECURITY_CONTEXT");
            Authentication auth = ctxt.getAuthentication();

            if(auth.getDetails() instanceof WebAuthenticationDetails){
                WebAuthenticationDetails dets = (WebAuthenticationDetails) auth.getDetails();
                System.out.println( "dets.getSessionId() is: " + dets.getSessionId() );
            }
            System.out.println("auth.getAuthorities() is: " + auth.getAuthorities() );
            System.out.println("auth.isAuthenticated() is: " + auth.isAuthenticated() );
        }
    }

    SecurityContext context = SecurityContextHolder.getContext();
    System.out.println("...........///////////// END OF DiagnoseSessionFilter.doFilterInternal() ///////////...........");
    fc.doFilter(req, res);

    }
}

---

Isolating the problem code:

---

The following combines and summarizes the diagnostic data from HttpSessionListener with the web browser's developer tools for the steps between the user clicking submit on the submit pin code view and the browser returning a rejection from the /oauth/token endpoint.

As you can see, there are two JSESSIONID values floating around. One of the values is correct, while the other value is not. The incorrect value gets passed into the request to /oauth/token, and causes rejection, even though the csrf passed is correct. Therefore, the solution to this problem will likely come from altering the steps below to stop placing the bad JSESSIONID in place of the good one:

1.) POST http://localhost:9999/uaa/secure/two_factor_authentication
    request headers:
        Referer: 9999/uaa/secure/two_factor_authentication
        Cookie: 
            JSESSIONID: ....95CB77     
                        ....918636
            XSRF-TOKEN: ....862a73
    filter chain:
        DiagnoseSessionFilter:
            request stuff:
                Cookie header:
                    JSESSIONID: ....95CB77
                                ....918636
                    XSRF-TOKEN: ....862a73
                request.getSession(false).getId(): ....95CB77
            session collector stuff:
                JSESSIONID: ....95CB77
                csrf: ....862a73
                SPRING_SECURITY_SAVED_REQUEST is null
            user details (from Authentication object with user/request
                JSESSIONID: ....ED927C
                Authenticated = true, with roles
        Complete the filter chain
        DiagnoseSessionFilter (again)
            request stuff:
                csrf attribute: ....862a73
                Cookie header: 
                    JSESSIONID: ....95CB77 
                                ....918636
                    XSRF-TOKEN: ....862a73
                request.getSession(false).getId(): 95CB77
            session collector stuff:
                JSESSIONID: ....95CB77
                csrf is: 862a73
                SPRING_SECURITY_SAVED_REQUEST is null
            user details (Authentication for user/session/request)
                JSESSIONID: ....ED927C
                Authenticated = true, with authorities
        POST/secure/two_factor_authenticationControllerMethod
            do some stuff
    response:
        Location: 9999/uaa/oauth/authorize?....
        XSRF-TOKEN: ....862a73

2.) GET http://localhost:9999/uaa/oauth/authorize?...
    request headers:
        Host: localhost:9999
        Referer: 9999/uaa/secure/two_factor_authentication
        Cookie: 
            JSESSIONID: ....95CB77    
                        ....918636
            XSRF-TOKEN: ....862a73
    FilterChain
        DiagnoseSessionFilter
            request stuff:
                Cookie header is: JSESSIONID: ....95CB77
                                              ....918636
                                  XSRF-TOKEN: ....862a73
                request.getSession(false).getId(): 95CB77
            session collector stuff: 
                JSESSIONID: ....95CB77
                csrf is: ....862a73
                SPRING_SECURITY_SAVED_REQUEST is: null
            user details (Authentication object with user/session/req)
                JSESSIONID: ....ED927C
                Authenticated = true with ALL roles.
        rest of filter chain
        TwoFactorAuthenticationFilter
            request stuff:
                csrf request attribute is: ....862a73
                cookie header:
                    JSESSIONID: ....95CB77
                                ....918636
                    XSRF-TOKEN: ....862a73
                request.getSession(false).getId() is: ....95CB77
                updateCsrf is: ....862a73
            response stuff:
                XSRF-TOKEN header (after manual update): ....862a73
        DiagnoseSessionFilter:
            request stuff:
                _csrf request attribute: ....862a73
                Cookie header:
                    JSESSIONID: ....95CB77
                                ....918636
                    XSRF-TOKEN: ....862a73
                    request.getSession(false).getId() is: ....95CB77
            session collector stuff: 
                JSESSIONID: ....95CB77
                csrf is: ....862a73
                SPRING_SECURITY_SAVED_REQUEST is: null
            user details (Authentication for user/session/request) 
                JSESSIONID: ....ED927C
                Authenticated is true, with ALL roles.
        CustomOAuth2RequestFactory
            request stuff:  
                _csrf request parameter is: ....862a73
                Cookie header: 
                    JSESSIONID: ....95CB77
                                ....918636
                    XSRF-TOKEN: ....862a73
                request.getSession(false).getId() is: ....95CB77
                updateCsrf: ....862a73
            response stuff:
                XSRF-TOKEN header: ....862a73
            session attribute printout
                csrf: ....862a73
                SPRING_SECURITY_CONTEXT (not printed, so don't know values)
    response:
        Location: 8080/login?code=myNwd7&state=f6b3Km
        XSRF-TOKEN: ....862a73

3.) GET http://localhost:8080/login?code=myNwd7&state=f6b3Km
    request headers:
        Host: localhost:8080
        Referer: 9999/uaa/secure/two_factor_authentication
        Cookie:  
            JSESSIONID: ....918636
            XSRF-TOKEN: ....862a73
    UiAppFilterChain:
        HttpSessionSecurityContextRepository
            creates new SPRING_SECURITY_CONTEXT to replace null one
        OAuth2ClientAuthenticationProcessingFilter (position 8 of 14)
            AuthorizationCodeAccessTokenProvider
                Retrieving token from 9999/uaa/oauth/token
    AuthServerFilterChain:
        DiagnoseSessionFilter
            request stuff:
                XSRF-TOKEN header is: null
                Cookie header is: null
                Set-Cookie header is: null
                referrer header is: null
                request.getSession(false) is: null
            session collector stuff:
                JSESSIONID: ....95CB77
                sessCsrf.getToken() is: 862a73
                SPRING_SECURITY_SAVED_REQUEST is: null
                Authenticated is true but with ONLY these roles: 
                    ROLE_HOBBIT, ROLE_TWO_FACTOR_AUTHENTICATION_ENABLED
            SecurityContextPersistenceFilter
                reports no HttpSession and no SPRING_SECURITY_CONTEXT
            CsrfFilter
                rejects request to /oauth/token due to no session % csrf

    response headers:
        Set-Cookie: 
            XSRF-TOKEN: ....527fbe
            X-Frame-Options: DENY

I will try to spend a little more time with this to further isolate the solution, given the number of points you are offering. But the above should substantially narrow the problem.

I am posting this before it is completely finished because your bounty period is about to expire.

Answer 2

Have you solved your issue? I have been looking around to find a full sample of 2FA together with spring-security-oauth2. It is great that you have posted your full concepts and the complete sources.

I tried your package and your issue can simply be resolved by changing just 1 line of code in your AuthserverApplication.java

@Override
    protected void configure(HttpSecurity http) throws Exception {
        // @formatter:off
        http
            .formLogin().loginPage("/login").permitAll()
        .and()
                .requestMatchers().antMatchers("/login", "/oauth/authorize", "/secure/two_factor_authentication", "/pincode")
        .and()
                .authorizeRequests().anyRequest().authenticated();
        // @formatter:on
    }

Your original configuration by passed the authentication chain of spring security which returned you a null object of authentication.

I would also recommend you to change the Bean creation of CustomOAuth2RequestFactory to the following which override all the OAuth2RequestFactory in the chain

@Bean
    public OAuth2RequestFactory customOAuth2RequestFactory(){
        return new CustomOAuth2RequestFactory(clientDetailsService);
    }

For the code you have added for handling the CSRF, you may just simply remove them, eg. the 2FA controller:

@Controller
@RequestMapping(TwoFactorAuthenticationController.PATH)
public class TwoFactorAuthenticationController {
    private static final Logger LOG = LoggerFactory.getLogger(TwoFactorAuthenticationController.class);
    public static final String PATH = "/secure/two_factor_authentication";
    public static final String AUTHORIZE_PATH = "/oauth/authorize";
    public static final String ROLE_TWO_FACTOR_AUTHENTICATED = "ROLE_TWO_FACTOR_AUTHENTICATED";

    private RedirectStrategy redirectStrategy = new DefaultRedirectStrategy();

    @RequestMapping(method = RequestMethod.GET)
    public String auth(HttpServletRequest request, HttpSession session, HttpServletResponse resp/*, ....*/) {
        System.out.println("-------- inside GET /secure/two_factor_authentication --------------");
        if (AuthenticationUtil.isAuthenticatedWithAuthority(ROLE_TWO_FACTOR_AUTHENTICATED)) {
            LOG.info("User {} already has {} authority - no need to enter code again", ROLE_TWO_FACTOR_AUTHENTICATED);
//            throw ....;
        }
        else if (session.getAttribute(CustomOAuth2RequestFactory.SAVED_AUTHORIZATION_REQUEST_SESSION_ATTRIBUTE_NAME) == null) {
//            LOG.warn("Error while entering 2FA code - attribute {} not found in session.", CustomOAuth2RequestFactory.SAVED_AUTHORIZATION_REQUEST_SESSION_ATTRIBUTE_NAME);
//          throw ....;
        }
        return "pinCode";
    }

    @RequestMapping(method = RequestMethod.POST)
    public String auth(FormData formData, HttpServletRequest req, HttpServletResponse resp,
                                            SessionStatus sessionStatus, Principal principal, Model model)
        throws IOException{

        if (formData.getPinVal()!=null) {
            if(formData.getPinVal().equals("5309")){
                AuthenticationUtil.addAuthority(ROLE_TWO_FACTOR_AUTHENTICATED);
                return "redirect:"+AUTHORIZE_PATH;
            };
        };

        return "pinCode";
    }
}

Please kindly let me know if you want a complete source codes after cleanup.