Can't connect to Kubernetes Deployment (App) when using the MetalLB load balancer. Attempting to configure floating IP so when one node fails and the workload moves across, I still know what the IP is when on the new node with new address.
Deployed:
- Raspbian on 4x Raspberry Pi 4s
- Enabled promiscuous mode on the ethernet ports
- Docker + Kubernetes
- CNI - Weave (set NO_MASQ_LOCAL = 1). [I have also set this up with Flannel with the same outcome]
- NGINX docker image
- MetalLB with Layer2 (Node network is 192.168.0.0/24, YAML for MetalLB has a small subnet allocated for the VIP 192.168.0.240/28)
Verified:
- Deployment (NGINX) works on CNI IP (10.44.0.1:80). Verified with curl to container IP from node hosting container.
- Deployment (NGINX) works on Nodeport using node IP (192.168.0.102). Verified with curl to node from laptop.
Deployment .yaml
metallb.yaml
apiVersion: v1
kind: ConfigMap
metadata:
namespace: metallb-system
name: config
data:
config: |
address-pools:
- name: my-ip-space
protocol: layer2
addresses:
- 192.168.0.240/28
nginx.yaml
apiVersion: apps/v1beta2
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1
ports:
- name: http
containerPort: 80
nginx-service -lb.yaml
apiVersion: v1
kind: Service
metadata:
name: nginx
labels:
app: nginx
spec:
selector:
app: nginx
type: LoadBalancer
ports:
- port: 80
name: http
protocol: TCP
targetPort: 80
Results
I think works:
- ARP requests to the VIP from the laptop I believe is working, MAC address of node hosting the container appears.
Doesn't work:
- Deploy load balancer which allocates a VIP 192.168.0.240 and port X, but when I try to connect to this from the worker node or my laptop unsuccessful. My laptop IP is in the same network. The node that this VIP is on can't perform the curl request either.
- I attempted making the loadbalancing subnet the /32 of the node with the deployment to see if it was an IP or ARP issue. When doing that I'm able to verify that the port allocated by MetalLB is open, but no response. This was done from my laptop.
Results from laptop
- ~ user$ curl 192.168.0.102:31854
- curl: (7) Failed to connect to 192.168.0.102 port 31854: Connection refused
ARP request from laptop .102 is the node, .240 is the VIP allocated by MetalLB
- ~ user$ arp 192.168.0.102
- ? (192.168.0.102) at dc:a6:32:b:fc:d on en0 ifscope [ethernet]
- ~ user$ arp 192.168.0.240
- ? (192.168.0.240) at dc:a6:32:b:fc:d on en0 ifscope [ethernet]
