2

I have a simple Node.js chat example which uses socket.io. I have tested this deployment with a docker container locally, and a web client succesfully connects to the websocket (socket.io). The problem arise when I try to deploy this on a Kubernetes cluster with an ingress Controller.

My question is, is there anything special socket.io specifically needs when using an ingress controller?

The error message I get is <<browser name>> Can't estabilish a connection to the server at ws://<<address>>/socket.io/EIO=3&transport=websocket

Below is the simple implementation of the server side

var io = socket(server);
io.on('connection', (socket)=>{
    console.log('made socket connection')
    socket.on('chat', (message)=>{
        message:message.value
    })

Below is the client:

var socket = io.connect("http://localhost:3002", {
    upgrade: false,
    transports: ['websocket'],
    secure: True
});

My ingress yaml file looks like this

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ws-ingress
  namespace: websocket-ns
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/websocket-services: "ws-service"
    nginx.ingress.kubernetes.io/proxy-read-timeout: '3600'
    nginx.ingress.kubernetes.io/proxy-send-timeout: '3600'
    nginx.ingress.kubernetes.io/server-snippet: |
      http {
        server {
            listen 3002;
          location = / {
              proxy_set_header Upgrade "websocket";
              proxy_http_version 1.1;
              proxy_set_header X-Forwarded-Host $http_host;
              proxy_set_header X-Forwarded-Proto $scheme;
              proxy_set_header X-Forwarded-For $remote_addr;
              proxy_set_header Host $host;
              proxy_set_header Connection "upgrade";
              proxy_cache_bypass $http_upgrade;
            }
          upstream nodes {
              hash $remote_addr consistent;
              server app01:3002;
            }
      }
spec:
  rules:
    - host: "$host_address"
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: ws-service
                port:
                  number: 80

Service yaml:

apiVersion: v1
kind: Service
metadata:
  name: ws-service
  namespace: websocket-ns
  labels:
    app: ws-service
spec:
  type: LoadBalancer
  selector:
    app: ws-app
  ports:
    - port: 80
      protocol: TCP
      targetPort: 3002

Server deployment file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ws-app
  namespace: websocket-ns
  labels:
    app: ws-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ws-app
  template:
    metadata:
      labels:
        app: ws-app
    spec:
      containers:
        - name: ws-app
          image: themuchy/socketexample
          ports:
            - containerPort: 3002
              name: ws-app
  • 1
    https://stackoverflow.com/a/58534529/5525824 – Harsh Manvar Aug 02 '21 at 12:51
  • not sure why you have added the `nginx.ingress.kubernetes.io/server-snippet` and upgrading the HTTP to websocket in config. That nginx controller will do it by own – Harsh Manvar Aug 02 '21 at 12:52
  • in ingress you traffic going into the `ws-service` however in annotation you mentioned another service name. – Harsh Manvar Aug 02 '21 at 12:53
  • 1
    @HarshManvar I used the same service name (ws-service) for ingress and in annotation, sorry, I just forgot to change the name in the annotation before I posted the question here (had been trying out a different solution). – Tinashe Muchabaiwa Aug 03 '21 at 09:05

1 Answers1

1

This one is working with one

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    certmanager.k8s.io/cluster-issuer: core-prod
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/proxy-read-timeout: "1800"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "1800"
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/secure-backends: "true"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/websocket-services: ws-service
    nginx.org/websocket-services: ws-service
  name: core-ingress
spec:
  rules:
  - host: test.io
    http:
      paths:
      - backend:
          serviceName: ws-service
          servicePort: 80
  tls:
  - hosts:
    - test.io
    secretName: core-prod

Nginx itself will upgrade the HTTP request and convert it to WebSocket. You don't have to add any annotation of the same.

Deployment and service, YAML configuration looking good however try removing the annotation server-snippet. This way traffic flow will be something like

LB > ingress (Connection upgrade) > service > pods
Harsh Manvar
  • 27,020
  • 6
  • 48
  • 102