Why do I get 502 when trying to authenticate

Question

I'm trying to implement authentication for my UI application I'm using https://github.com/joaojosefilho/vuejsOidcClient which I succesffully configured to work with gluu auth server

Now I wanted to try with keycloak what happens is that flow is started with which looks ok

http://auth.solidsense.tk/auth/realms/master/protocol/openid-connect/auth?client_id=console&redirect_uri=http%3A%2F%2Fportal.solidsense.tk%2Fcallback.html&response_type=id_token%20token&scope=openid%20profile%20permission%20address%20roles&state=f60457ff91c84ac5b49179bc4b47cd8d&nonce=3753dd32c3ea4f8087608382c2eef932

and keycloak login page is presented but after entering credentials on submit I get error 502 for the following request ( if incorrect credentials are entered proper error messages is shown )

http://auth.solidsense.tk/auth/realms/master/login-actions/authenticate?session_code=2xol67J32yIFVfDfrN0DmIT1sdzfzoeo1dg_kPZrYeY&execution=499b2298-5c08-4521-8993-b23eb253f0a4&client_id=console&tab_id=cHj4HozAr-Q

this is the exception that I see in the log

08:03:26,164 DEBUG [io.undertow.request.io] (default task-1) UT005013: An IOException occurred: java.io.IOException: Connection reset by peer                                                
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)                                                                                                                                
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)                                                                                                                        
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)                                                                                                                           
        at sun.nio.ch.IOUtil.read(IOUtil.java:197)                                                                                                                                           
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)                                                                                                                     
        at org.xnio.nio.NioSocketConduit.read(NioSocketConduit.java:289)                                                                                                                     
        at org.xnio.conduits.ConduitStreamSourceChannel.read(ConduitStreamSourceChannel.java:127)                                                                                            
        at io.undertow.util.ConnectionUtils.doDrain(ConnectionUtils.java:96)                                                                                                                 
        at io.undertow.util.ConnectionUtils.cleanClose(ConnectionUtils.java:74)                                                                                                              
        at io.undertow.server.protocol.http.HttpReadListener.exchangeComplete(HttpReadListener.java:368)                                                                                     
        at io.undertow.server.protocol.http.HttpServerConnection.exchangeComplete(HttpServerConnection.java:232)                                                                             
        at io.undertow.server.HttpServerExchange.invokeExchangeCompleteListeners(HttpServerExchange.java:1279)                                                                               
        at io.undertow.server.HttpServerExchange.terminateResponse(HttpServerExchange.java:1563)                                                                                             
        at io.undertow.server.Connectors.terminateResponse(Connectors.java:147)                                                                                                              
        at io.undertow.server.protocol.http.ServerFixedLengthStreamSinkConduit.channelFinished(ServerFixedLengthStreamSinkConduit.java:58)                                                   
        at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.exitFlush(AbstractFixedLengthStreamSinkConduit.java:316)                                                                
        at io.undertow.conduits.AbstractFixedLengthStreamSinkConduit.flush(AbstractFixedLengthStreamSinkConduit.java:234)                                                                    
        at org.xnio.conduits.ConduitStreamSinkChannel.flush(ConduitStreamSinkChannel.java:162)                                                                                               
        at io.undertow.channels.DetachableStreamSinkChannel.flush(DetachableStreamSinkChannel.java:119)                                                                                      
        at org.xnio.channels.Channels.flushBlocking(Channels.java:63)                                                                                                                        
        at io.undertow.servlet.spec.ServletOutputStreamImpl.close(ServletOutputStreamImpl.java:618)                                                                                          
        at io.undertow.servlet.spec.HttpServletResponseImpl.closeStreamAndWriter(HttpServletResponseImpl.java:486)                                                                           
        at io.undertow.servlet.spec.HttpServletResponseImpl.responseDone(HttpServletResponseImpl.java:575)                                                                                   
        at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:351)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81)
        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
        at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)

        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138)
        at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135)
        at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48)
        at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43)
        at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
        at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502)
        at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272)
        at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81)
        at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104)
        at io.undertow.server.Connectors.executeRootHandler(Connectors.java:364)
        at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
        at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
        at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
        at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
        at java.lang.Thread.run(Thread.java:748)

08:03:27,805 DEBUG [org.keycloak.transaction.JtaTransactionWrapper] (Timer-2) new JtaTransactionWrapper
08:03:27,806 DEBUG [org.keycloak.transaction.JtaTransactionWrapper] (Timer-2) was existing? false
08:03:27,811 DEBUG [org.keycloak.models.sessions.infinispan.changes.sessions.PersisterLastSessionRefreshStore] (Timer-2) Updating 0 userSessions with lastSessionRefresh: 1557813747
08:03:27,812 DEBUG [org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorImpl] (Timer-2) Hibernate RegisteredSynchronization successfully registered with JTA pla
tform

If i try to access /auth/realms/master/account there is no problem but that uses authorization code flow

http://auth.solidsense.tk/auth/realms/master/protocol/openid-connect/auth?client_id=account&redirect_uri=http%3A%2F%2Fauth.solidsense.tk%2Fauth%2Frealms%2Fmaster%2Faccount%2Flogin-redirect&state=0%2F2758cd0b-d1c0-4fd6-8c0f-a8aa1916aad6&response_type=code&scope=openid

keycloak is running behind nginx

  listen 80;
  listen [::]:80;
  server_name auth.solidsense.tk;
  location /{ 
    proxy_set_header Host $host; 
    proxy_set_header X-Real-IP  $remote_addr; 
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://localhost:9080;

I expect that after the clicking the submit button and after keycloak validates username and password to get redirected to my applications redirect_uri

What do you see in nginx error log? – Danila Vershinin Aug 14 '19 at 20:22 — Danila Vershinin, Aug 14 '19 at 20:22
I have the similar issue. Did you find out the reason? – Kaman Wu Nov 21 '20 at 21:53 — Kaman Wu, Nov 21 '20 at 21:53

score 35 · Answer 1 · answered May 13 '21 at 03:54

35

I got this same 502 error.

Nginx error log was reporting "*1 upstream sent too big header while reading response header from upstream,". Increasing the buffer size fixed the issue.

Add these directives to the http block in nginx.conf

proxy_buffer_size   128k;
proxy_buffers   4 256k;
proxy_busy_buffers_size   256k;

answered May 13 '21 at 03:54

Seamus

699
1
6
12

2

Thanks, this was a lifesaver! Not all 3 directives are always necessary, however, this article helped us tune it for our specific situation: https://www.getpagespeed.com/server-setup/nginx/tuning-proxy_buffer_size-in-nginx. – tobinibot Apr 11 '23 at 13:13

score 32 · Answer 2 · answered Aug 17 '21 at 02:54

32

For those of you that are dealing with Nginx Kubernetes ingress:

"nginx.ingress.kubernetes.io/proxy-buffer-size": "128k"

More info here: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#proxy-buffer-size

answered Aug 17 '21 at 02:54

Matteo

2,256
26
42

Ladislav Račák · Answer 3 · 2021-01-22T20:26:05.390

10

The problem might be in Nginx configuration. Try setting proxy_buffers to a higher number. e.g.:

listen 80;
  listen [::]:80;
  server_name auth.solidsense.tk;
  location /{ 
    proxy_set_header Host $host; 
    proxy_set_header X-Real-IP  $remote_addr; 
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_buffers 4 16k;
    proxy_pass http://localhost:9080;
    ...

edited Jan 22 '21 at 20:26

answered Aug 14 '19 at 14:15

Ladislav Račák

189
1
6

score 4 · Answer 4 · answered Oct 04 '22 at 10:09

Root cause is low nginx proxy buffer size. You need to increase it, for example 128k. If you are using kubernetes ingress like me, you can use the following settings.

nginx.ingress.kubernetes.io/proxy-buffer-size: "128k"

ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-keycloak
  annotations:
    nginx.ingress.kubernetes.io/proxy-buffer-size: "128k"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - keycloak.mesutpiskin.com
    secretName:  wildcard-fimple
  rules:
  - host: keycloak.mesutpiskin.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: test-keycloak
            port:
              number: 8080

score 2 · Answer 5 · answered Jan 03 '23 at 11:46

2

for anyone deploying with the official nginx-ingress chart from https://artifacthub.io/packages/helm/nginx/nginx-ingress, the annotation is nginx.org/proxy-buffer-size: "128k"

answered Jan 03 '23 at 11:46

Julius á Rógvi Biskopstø

21
2

score 0 · Answer 6 · answered Aug 17 '23 at 19:25

0

nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" is what did it for me

answered Aug 17 '23 at 19:25

bobslaede

31
2

score 0 · Answer 7 · answered Aug 23 '23 at 02:51

For those who are using Ingress Controller on Kubernetes: Update Config Map that corresponds to the Ingress Controller, in my case ingress-nginx-controller, as shown below

apiVersion: v1
kind: ConfigMap
metadata:
.......
.....
data:
  .........
  proxy-buffer-size: 128k
  proxy_buffers: 4 256k
  proxy_busy_buffers_size: 256k

Why do I get 502 when trying to authenticate

7 Answers7

Linked