EKS ingress-nginx and NLB with https redirect

Question

I'm having issues with the nlb lately, it was quite an adventure to have nlb with https termination on the lb working with a redirection http=>https and an ingress-nginx on EKS.

Now, I want to have the X-Forwarded headers passed to the pod, but that breaks the http=>https redirection, I get a 400 on http requests.

On the service, I tried to put the service with http or tcp protocol, same thing.

Adding the service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*" header to the service, activates the proxy protocol v2 on all targets, and activating use-proxy-protocol: 'true' in the configmap for nginx breaks the http-snippet with the 308 redirection:

http-snippet: |
    server {
      listen 2443;
      return 308 https://$host$request_uri;
    }

Does anyone has a way to make it so that it can use the nlb with all the good header and the redirect working?

EDIT at comment request adding full working config

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/version: 0.41.0
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
data:
  http-snippet: |
    server {
      listen 2443 proxy_protocol;
      return 308 https://$host$request_uri;
    }
  proxy-real-ip-cidr: 10.4.0.0/16
  use-forwarded-headers: 'true'
  use-proxy-protocol: 'true'
  compute-full-forwarded-for: 'true'

Where did you get `ingress-nginx` from? https://kubernetes.github.io/ingress-nginx/deploy/ ? If so then to enable HTTPS redirect you only have to configure TLS in an Ingress resource: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#server-side-https-enforcement-through-redirect — anemyte, Jan 07 '21 at 14:49
Please read the title, I'm specifically asking in the use case where you have an NLB not a CLB or ALB. — night-gold, Jan 07 '21 at 14:52
I've read it again and I still don't get why do you use that http-snippet for redirects when nginx do that automatically if TLS configured in an ingress resource. I use NLB and nginx ingress in production myself and I didn't have to do a thing to get what you are trying to achieve. — anemyte, Jan 07 '21 at 15:06
I see now. Hope this will help you https://github.com/kubernetes/kubernetes/issues/73297#issuecomment-601787548 — anemyte, Jan 07 '21 at 15:13
The ssl part is working, the issue is that the http=>https redirect is broken when trying to have the header passed to the pod. — night-gold, Jan 07 '21 at 15:15
Looking at the official guide (https://docs.nginx.com/nginx/admin-guide/load-balancer/using-proxy-protocol/#configuring-nginx-to-accept-the-proxy-protocol) it seems `listen` directive should contain `proxy_protocol` for NGINX to accepts proxy protocol headers. Could you try changing `listen 2443;` for `listen 2443 proxy_protocol;`? — anemyte, Jan 07 '21 at 16:04
@anemyte That's it... It's obvious but I thought that it was doing it was doing it as long as you activated it in the configmap... You can put that in an answer and I will validate :) — night-gold, Jan 07 '21 at 16:29
@night-gold mind sharing the full working example? it will be super useful for us — Amit Baranes, Aug 20 '21 at 09:00

score 1 · Accepted Answer · answered Jan 07 '21 at 16:48

1

To conclude our comment discussion with @night-gold, to make NGINX to accept proxy protocol you have to specifically mention that in listen directive:

http {
    #...
    server {
        listen 80   proxy_protocol;
        listen 443  ssl proxy_protocol;
        #...
    }
}

Check out NGINX guide for more.

answered Jan 07 '21 at 16:48

anemyte

17,618
1
24
45

2

Just a little precision, the directive is activated in the configmap using use-proxy-protocol: 'true' to have it globally. But it's not taken into account inside the http-snippet of the configmap. – night-gold Jan 07 '21 at 16:50

johanv26 · Answer 2 · 2022-09-22T16:30:14.133

Actually I tried to use that config with the HTTP snippet. But it didn't work. So searching about it a find the PR that solves the redirect with that config. The PR also exposes to the solution Port 2443 with the configuration tohttps in order to make the redirect work. I based on that one and work it for me.

https://github.com/kubernetes/ingress-nginx/pull/5374

https://github.com/kubernetes/ingress-nginx/pull/5374/files#diff-885b46a1b162f530aa95239e8c3adf9887a4ce863b443f49f06368011a4259ddR390-R393

score -2 · Answer 3 · answered Sep 21 '21 at 14:09

In addition to this answer don't forget to add annotation of Proxy Protocol to service.

https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/service/nlb/#protocols

Here is my values in helm chart

helm upgrade \
      ingress-nginx ingress-nginx/ingress-nginx \
      --namespace ingress-nginx \
      --set controller.service.type=LoadBalancer \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-type"=nlb \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-nlb-target-type"=ip \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-cross-zone-load-balancing-enabled"=true \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-type"=external \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-name"="nginx-ingress" \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-proxy-protocol"="*" \
      --set controller.service.annotations."service\.beta\.kubernetes\.io/aws-load-balancer-scheme"="internet-facing" \
      --set controller.config."proxy-real-ip-cidr"="xx.xx.xx.xx/xx" \
      --set controller.config."use-forwarded-headers"="true" \
      --set controller.config."use-proxy-protocol"="true" \
      --set controller.config."compute-full-forwarded-for"="true" \
      --set controller.config."http-snippet"="
server{
  listen 2443;
  return 308 https://\$host\$request_uri;
}" \
      --dry-run

Helm conf is irrelevant in this case, please see question and you can see that I put the conf lines (not helm) — night-gold, Sep 24 '21 at 16:54

EKS ingress-nginx and NLB with https redirect

3 Answers3