3

I'm using AWS Load Balancer Controller for setup AWS NLB with TLS termination for Grpc service (grpc-dotnet implementation) running in EKS. Traffic between NLB and Pod in EKS is unencrypted. I also tried to set it up in UI but it ends with same result. The k8s service with annotation for NLB configuration:

apiVersion: v1
kind: Service
metadata:
    annotations:
        service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: 'true'
        service.beta.kubernetes.io/aws-load-balancer-healthcheck-path: /health
        service.beta.kubernetes.io/aws-load-balancer-internal: 'true'
        service.beta.kubernetes.io/aws-load-balancer-type: nlb-ip
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: tcp
        service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: "ELBSecurityPolicy-TLS13-1-2-2021-06"
        service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:eu-west-1:account:certificate/my-cert-arn"
        service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "9090"
        # service.beta.kubernetes.io/aws-load-balancer-alpn-policy: HTTP2Preferred
    name: grpc-api-svc
    namespace: default
spec:
    ports:
        - 
            port: 9090
            protocol: TCP
            targetPort: 9090
    selector:
        app: grpc-api
    type: LoadBalancer

The configuration route traffic from NLB to my Grpc Api, but requests end with error:

{
  "created": "@1678456920.910000000",
  "description": "Failed to pick subchannel",
  "file": "src/core/ext/filters/client_channel/client_channel.cc",
  "file_line": 5391,
  "referenced_errors": [
    {
      "created": "@1678456920.909000000",
      "description": "failed to connect to all addresses",
      "file": "src/core/ext/filters/client_channel/lb_policy/pick_first/pick_first.cc",
      "file_line": 398,
      "grpc_status": 14
    }
  ]
}

When I remove TLS termination (SSL part) it works well.

I also tried to add ALPN service.beta.kubernetes.io/aws-load-balancer-alpn-policy: HTTP2Preferred with same result.

EDIT: Python service work well, the issue appears only when I'm using .Net Grpc Server (grpc-dotnet)

Root Cause: NLB does terminate TLS but keep in header https schema. It seems the .net server is more strict and Kestrel ignore these requests.

Solution You have to set in Kestrel options

options.AllowAlternateSchemes = true
Holden Cauldfield
  • 51
  • 4

0 Answers0