nginx-ingress: Too many redirects when force-ssl is enabled

Question

I am setting up my first ingress in kubernetes using nginx-ingress. I set up the ingress-nginx load balancer service like so:

{
  "kind": "Service",
  "apiVersion": "v1",
  "metadata": {
    "name": "ingress-nginx",
    "namespace": "...",
    "labels": {
      "k8s-addon": "ingress-nginx.addons.k8s.io"
    },
    "annotations": {     
      "service.beta.kubernetes.io/aws-load-balancer-backend-protocol": "tcp",
      "service.beta.kubernetes.io/aws-load-balancer-proxy-protocol": "*",
      "service.beta.kubernetes.io/aws-load-balancer-ssl-cert": "arn....",
      "service.beta.kubernetes.io/aws-load-balancer-ssl-ports": "443"
    }
  },
  "spec": {
    "ports": [
      {
        "name": "http",
        "protocol": "TCP",
        "port": 80,
        "targetPort": "http",
        "nodePort": 30591
      },
      {
        "name": "https",
        "protocol": "TCP",
        "port": 443,
        "targetPort": "http",
        "nodePort": 32564
      }
    ],
    "selector": {
      "app": "ingress-nginx"
    },
    "clusterIP": "...",
    "type": "LoadBalancer",
    "sessionAffinity": "None",
    "externalTrafficPolicy": "Cluster"
  },
  "status": {
    "loadBalancer": {
      "ingress": [
        {
          "hostname": "blablala.elb.amazonaws.com"
        }
      ]
    }
  }
}

Notice how the https port has its targetPort property pointing to port 80 (http) in order to terminate ssl at the load balancer.

My ingress looks something like this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata: 
  name: something
  namespace: ...
  annotations:
    ingress.kubernetes.io/ingress.class: "nginx"
    ingress.kubernetes.io/force-ssl-redirect: "true"
spec:
  rules:
    - host: www.exapmle.com
      http:
        paths:
         - path: /
           backend:
            serviceName: some-service
            servicePort: 2100

Now when I navigate to the url I get a Too many redirects error. Something that is confusing me is that when I add the following header "X-Forwarded-Proto: https" I get the expected response (curl https://www.example.com -v -H "X-Forwarded-Proto: https").

Any ideas how I can resolve the issue?

P.S. this works just fine with ingress.kubernetes.io/force-ssl-redirect: "false" and it doesn't seem that there are any extraneous redirects.

score 14 · Answer 1 · answered Nov 01 '21 at 08:55

Adding another cause for the Too many redirects error.

While working with ingress-nginx as an ingress controller in front of some k8s services.

One of the services (ArgoCD in my case) handled TLS termination by itself and always redirects HTTP requests to HTTPS.

The problem is that the nginx ingress controller also handled TLS termination and communicates with the backend service with HTTP then the result is that the ArgoCD's server always responding with a redirects to HTTPS which is the cause for the multiple redirects.

Any attempts to pass relevant values to the ingress annotations below will not help:

annotations:
  nginx.ingress.kubernetes.io/ssl-redirect: false/true
  nginx.ingress.kubernetes.io/backend-protocol: "HTTP"/"HTTPS"

The solution was to ensure that the service doesn't handle TLS by passing --insecure flag to the argocd-server deployment:

spec:

  template:
    spec:
      containers:
      - name: argocd-server
        command:
        - argocd-server
        - --repo-server
        - argocd-repo-server:8081
        - --insecure # <-- Here

Commented on 12/23/2021. Nothing is working but yours. Thank you soooo much — Johnny, Dec 23 '21 at 23:43
I'm using ArgoCD as well, which needs ssl-passthrough configured, as per docs: https://argo-cd.readthedocs.io/en/stable/operator-manual/ingress/#option-1-ssl-passthrough — tschale, Oct 14 '22 at 11:29
the official documentation of argocd, # If you encounter a redirect loop or are getting a 307 response code # then you need to force the nginx ingress to connect to the backend using HTTPS. # nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" — Fulvio, Jun 30 '23 at 02:30

Anton Kostenko · Accepted Answer · 2018-04-27T08:18:34.600

10

That is a known issue with the annotation for SSL-redirection in combination with proxy-protocol and termination of SSL connections on ELB.

Question about it was published on GitHub and here is a fix from that thread:

You should create a custom ConfigMap for an Nginx-Ingress instead of using force-ssl-redirect annotation like the following:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: ingress-nginx
  name: nginx-ingress-configuration
  namespace: <ingress-namespace>
data:
  ssl-redirect: "false"
  hsts: "true"
  server-tokens: "false"
  http-snippet: |
    server {
      listen 8080 proxy_protocol;
      server_tokens off;
      return 301 https://$host$request_uri;
    }

That configuration will create an additional listener with a simple redirection to https.

Then, apply that ConfigMap to your ingress-controller, add NodePort 8080 to its container definition and to the Service.
Now, you can point the port 80 of your ELB to port 8080 of the Service.

With that additional listener, it will work.

edited Apr 27 '18 at 08:18

answered Apr 16 '18 at 12:32

Anton Kostenko

8,200
2
30
37

Why listen 8000 and NodePort 8080? – Latrova Apr 26 '18 at 20:33
1

Mistype. Thanks, I set 8080 in the config. – Anton Kostenko Apr 27 '18 at 08:19
About step 2: Does it have to be a `NodePort`? Can I have a `ClusterIP` instead? – Latrova Apr 27 '18 at 13:10
Depends of your configuration, actually. The point is to make it forward requests from clients to that port from `http` port. That is just a small redirection server, which will forward client from `http://_anything_` to `https://the_same_anything_`. Nothing more. – Anton Kostenko Apr 27 '18 at 13:36
How can we do this on ingress level, for specific ones only. Instead of configmap for the whole ingress controller??? – Arash Mar 02 '20 at 23:41
how does one apply this with helm install using the public helm chart? – uberrebu Jun 22 '21 at 15:13
doesn't work for me – SledgeHammer Aug 20 '23 at 01:33

score 4 · Answer 3 · answered Dec 10 '20 at 20:38

4

I had to add these annotations to make it work without changing the ingress-controller:

    annotations:
      kubernetes.io/ingress.class: nginx-ingress-internal # <- AWS NLB
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
      nginx.ingress.kubernetes.io/configuration-snippet: |
        if ($http_x_forwarded_proto = 'http') {
          return 301 https://$host$request_uri;
        }

answered Dec 10 '20 at 20:38

dpolicastro

1,379
1
14
22

1

best answer as per my env – Abdennour TOUMI Mar 06 '21 at 23:44
1

This works fine, and best suits for all environments – Vishesh Kumar Singh Mar 31 '21 at 07:24
doesn't work either. – SledgeHammer Aug 20 '23 at 01:39

score 2 · Answer 4 · answered Oct 06 '20 at 11:20

Another approach that worked for my environment (k8s v1.16.15, rancher/nginx-ingress-controller:nginx-0.32.0-rancher1):

apiVersion: v1
data:
  compute-full-forwarded-for: "true"
  use-forwarded-headers: "true"
kind: ConfigMap
metadata:
  labels:
    app: ingress-nginx
  name: nginx-configuration
  namespace: ingress-nginx

This worked with the force-ssl-redirect on the ingress of the application. It seems that the ingress-controller does not use the X-Forwarded-Proto header from the ELB out of the box.

score 0 · Answer 5 · answered Dec 29 '22 at 11:58

I had this issues in Keycloak setup via helm chart as well. The SSL termination is done on ELB so to fix it . I made the following changes in helm values.

ingress:
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "false"

This helped me fixed it.

score 0 · Answer 6 · answered Apr 02 '23 at 11:18

0

Adding this annotation: nginx.ingress.kubernetes.io/backend-protocol: HTTPS

fixed it for me. full example here: https://github.com/argoproj/argoproj-deployments/blob/master/argo-workflows/resources/argo-server-ingress.yaml

answered Apr 02 '23 at 11:18

Devqxz

51
1
6

score -2 · Answer 7 · answered Mar 02 '23 at 18:49

I had this same issue. In my case, this configs are working:

nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/enable-underscores-in-headers: "true"

nginx-ingress: Too many redirects when force-ssl is enabled

7 Answers7

Linked