When I replicate the application on more than one pod, the web-app will return a Http 504 while using a NGINX load-balancer.
The NGINX load-balancer is outside of the K8 cluster and acts as a reverse-proxy + load-balancer. Hence, NGINX will forward the requests to one node, hosting the web-app container. Important: I don't want the NGINX host to be part of the cluster. (As long as it can be prevented)
upstream website {
ip_hash;
server 1.1.1.1:30300;
#server 2.2.2.2:30300;
}
server {
listen 443 ssl http2;
server_name example.com;
location / {
proxy_pass http://website;
proxy_cache off;
proxy_buffering off;
proxy_read_timeout 1d;
proxy_connect_timeout 4;
proxy_send_timeout 1d;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $http_connection;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
This config does work, if, and only if, the app has been published to the 1.1.1.1 node only. If I replicate the web-app to 2.2.2.2 as well, the snippet above will already lead to a 504, even thought 2.2.2.2 is still commented out. Commenting the 2.2.2.2 in, won't change anything.
As far as I understood, the NodePort is a public-available port, mapping to an internal port. (Called port) Hence, NodePort 30300 will be forwarded to 2000, which is also my targetport the web-app listens on. Upon replication the second pod will also host the web-app (+ microservices) and expose itself to NodePort 30300. So we do have two NodePorts 30300 within our k8 network and I guess this might lead to confusion and routing issues.
apiVersion: apps/v1
kind: Deployment
metadata:
name: swiper-web
namespace: default
spec:
replicas: 2
selector:
matchLabels:
app: swiper
template:
metadata:
labels:
app: swiper
spec:
containers:
- name: swiper-web-app-example
image: docker.example.com/swiper.web.app.webapp:$(Build.BuildId)
ports:
- containerPort: 2000
resources:
limits:
memory: "2.2G"
cpu: "0.6"
- name: swiper-web-api-oauth
image: docker.example.com/swiper.web.api.oauth:$(Build.BuildId)
ports:
- containerPort: 2010
resources:
limits:
memory: "100M"
cpu: "0.1"
imagePullSecrets:
- name: regcred
dnsPolicy: "None"
dnsConfig:
nameservers:
- 8.8.8.8
---
apiVersion: v1
kind: Service
metadata:
name: swiper-web-service
namespace: default
spec:
type: NodePort
selector:
app: swiper
ports:
- name: swiper-web-app-example
port: 2000
nodePort: 30300
- name: swiper-web-api-oauth
port: 2010
Edit:
Adding externalTrafficPolicy: Local to the swiper-web-service solves the issue. Both endpoints are now reachable. But the load-balacing of the other microservices is now disabled.
