Kubernetes, Flannel and exposing services

Question

I have a kubernetes setup running nicely, but I can't seem to expose services externally. I'm thinking my networking is not set up correctly:

kubernetes services addresses: --service-cluster-ip-range=172.16.0.1/16

flannel network config: etcdctl get /test.lan/network/config {"Network":"172.17.0.0/16"}

docker subnet setting: --bip=10.0.0.1/24

Hostnode IP: 192.168.4.57

I've got the nginx service running and I've tried to expose it like so:

[root@kubemaster ~]# kubectl get pods
NAME          READY     STATUS    RESTARTS   AGE
nginx-px6uy   1/1       Running   0          4m
[root@kubemaster ~]# kubectl get services
NAME         LABELS                                    SELECTOR    IP(S)           PORT(S)    AGE
kubernetes   component=apiserver,provider=kubernetes   <none>      172.16.0.1      443/TCP    31m
nginx        run=nginx                                 run=nginx   172.16.84.166   9000/TCP   3m

and then I exposed the service like this:

kubectl expose rc nginx --port=9000 --target-port=9000 --type=NodePort
NAME      LABELS      SELECTOR    IP(S)     PORT(S)    AGE
nginx     run=nginx   run=nginx             9000/TCP   292y

I'm expecting now to be able to get to the nginx container on the hostnodes IP (192.168.4.57) - have I misunderstood the networking? If I have, can explanation would be appreciated :(

Note: This is on physical hardware with no cloud provider provided load balancer, so NodePort is the only option I have, I think?

score 2 · Accepted Answer · answered Dec 24 '15 at 12:55

So the issue here was that there's a missing piece of the puzzle when you use nodePort.

I was also making a mistake with the commands.

Firstly, you need to make sure you expose the right ports, in this case 80 for nginx:

kubectl expose rc nginx --port=80 --type=NodePort

Secondly, you need to use kubectl describe svc nginx and it'll show you the NodePort it's assigned on each node:

[root@kubemaster ~]# kubectl describe svc nginx
Name:           nginx
Namespace:      default
Labels:         run=nginx
Selector:       run=nginx
Type:           NodePort
IP:         172.16.92.8
Port:           <unnamed>   80/TCP
NodePort:       <unnamed>   32033/TCP
Endpoints:      10.0.0.126:80,10.0.0.127:80,10.0.0.128:80
Session Affinity:   None
No events.

You can of course assign one when you deploy, but I was missing this info when using randomly assigned ports.

score 1 · Answer 2 · answered Dec 18 '15 at 18:33

1

yes, you would need to use NodePort. When you hit the service, the destPort should be equal to NodePort. The destIP for the service should be considered local by the nodes. E.g. you could use the hostIP of one of the nodes..

A load-balancer helps because it would handle situations where your node went down, but other nodes could still process the service..

answered Dec 18 '15 at 18:33

Abhishek Shah

76
2

Thanks for replying Abhishek. I was expecting that to be the case, but I can't even get the nginx to load on the host (that's the node, right, in this case?) to load the hello world page. `telnet kube1.leebriggs.lan 9000 Trying 192.168.4.56... telnet: connect to address 192.168.4.56: Connection refused` – jaxxstorm Dec 18 '15 at 18:41
@Frap Make sure you enable port 9000 in the firewall. – George Dec 21 '15 at 20:50

score 1 · Answer 3 · answered Jan 05 '16 at 22:14

if you're running a cluster on bare metal or not at a provider that provides the load balancer, you can also define the port to be a hostPort on your pod

you define your container, and ports

containers:
- name: ningx
  image: nginx
  ports:
  - containerPort: 80
    hostPort: 80
    name: http

this will bind the container to the host networking and use the port defined.

The 2 limitations here are obviously: 1) You can only have one of these pods on each host maximum. 2) The IP is the host IP of the node it binds to

this is essentially how the cloud provider load balancers work in a way.

Using the new DaemonSet features, it's possible to define what node the pod will land on and fix the IP. However that necessarily impair the high availability aspect, but at some point there is not much choice as DNS load balancing will not avoid forwarding to a dead nodes

Kubernetes, Flannel and exposing services

3 Answers3