17

I'd like to use Microk8s for playing around Spring Cloud Data Flow, so I installed Multipass on my Windows 10, and in a virtual machine I run an Ubuntu where Microk8s is installed. I'd like to achieve that, all containers run on Ubuntu, which is a headless Virtualbox VM managed by Multipass, but I'd like to be able to connect to the containers from Windows.

The topology looks like this:

Windows 10:

  • 192.168.1.107 provided by my home router

Ubuntu which runs in Virtualbox

  • 192.168.52.1 provided by, I believe, Windows for Virtualbox (Virtualbox Host-Only Network)

If I log in Ubuntu ifconfig says the following:

multipass@scdf:~/scdf/spring-cloud-dataflow$ ifconfig                                                                                                                               
cbr0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500                                                                                                                                  
        inet 10.1.1.1  netmask 255.255.255.0  broadcast 0.0.0.0                                                                                                                             
        inet6 fe80::c802:9fff:fea2:93f1  prefixlen 64  scopeid 0x20<link>                                                                                                                   
        ether ca:02:9f:a2:93:f1  txqueuelen 1000  (Ethernet)                                                                                                                                
        RX packets 19890  bytes 6628126 (6.6 MB)                                                                                                                                            
        RX errors 0  dropped 0  overruns 0  frame 0                                                                                                                                         
        TX packets 21413  bytes 13988665 (13.9 MB)  

        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0                                                                                                                  
enp0s3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.0.2.15  netmask 255.255.255.0  broadcast 10.0.2.255
        inet6 fe80::a00:27ff:feba:30bb  prefixlen 64  scopeid 0x20<link>
        ether 08:00:27:ba:30:bb  txqueuelen 1000  (Ethernet)
        RX packets 208972  bytes 273941063 (273.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 24249  bytes 2084939 (2.0 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 157483  bytes 40950239 (40.9 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 157483  bytes 40950239 (40.9 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

microk8s.kubectl get all says the following, it's been executed in Ubuntu:

multipass@scdf:~/scdf/spring-cloud-dataflow$ microk8s.kubectl get all
NAME                                          READY   STATUS    RESTARTS   AGE
pod/default-http-backend-5d5ff5d4f5-6ttvd     1/1     Running   1          29m
pod/kafka-broker-64445d8596-88rl6             1/1     Running   24         98m
pod/kafka-zk-77bdb5bcbb-qjxfh                 1/1     Running   4          98m
pod/mysql-bdd598966-nknwk                     0/1     Pending   0          87m
pod/nginx-ingress-microk8s-controller-dh28n   1/1     Running   1          29m


NAME                           TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/default-http-backend   ClusterIP      10.152.183.57    <none>        80/TCP                       29m
service/kafka                  ClusterIP      10.152.183.194   <none>        9092/TCP                     98m
service/kafka-zk               ClusterIP      10.152.183.80    <none>        2181/TCP,2888/TCP,3888/TCP   98m
service/kubernetes             ClusterIP      10.152.183.1     <none>        443/TCP                      100m
service/mysql                  ClusterIP      10.152.183.113   <none>        3306/TCP                     87m
service/scdf-server            LoadBalancer   10.152.183.122   <pending>     80:32548/TCP                 80m
service/skipper                LoadBalancer   10.152.183.163   <pending>     80:30955/TCP                 81m

NAME                                               DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/nginx-ingress-microk8s-controller   1         1         1       1            1           <none>          29m

NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/default-http-backend   1/1     1            1           29m
deployment.apps/kafka-broker           1/1     1            1           98m
deployment.apps/kafka-zk               1/1     1            1           98m
deployment.apps/mysql                  0/1     1            0           87m
deployment.apps/scdf-server            0/1     0            0           80m
deployment.apps/skipper                0/1     0            0           82m

NAME                                              DESIRED   CURRENT   READY   AGE
replicaset.apps/default-http-backend-5d5ff5d4f5   1         1         1       29m
replicaset.apps/kafka-broker-64445d8596           1         1         1       98m
replicaset.apps/kafka-zk-77bdb5bcbb               1         1         1       98m
replicaset.apps/mysql-bdd598966                   1         1         0       87m
replicaset.apps/scdf-server-6988d7795f            1         0         0       80m
replicaset.apps/skipper-5957946bf5                1         0         0       82m

As you can see there are two <Pending> status for external IP addresses. I believe those should be exposed in order to be able to connect to.

I did what I found in this help, but I couldn't reach that the server gets external IP.

What I'm doing here wrong? Is Microk8s able to expose its containers?

AndrasCsanyi
  • 3,943
  • 8
  • 45
  • 77

2 Answers2

26

You are right, if you want to connect to your cluster from outside, you have to expose service as LoadBalancer or NodePort.

Unfortunately, as you are not using any cloud provider which support LoadBalancer (like AWS or GCP which provides External-IPs) you will be not able to expose service as LoadBalancer (service stuck on Pending state). As describe here:

LoadBalancer: Exposes the Service externally using a cloud provider’s load balancer. NodePort and ClusterIP Services, to which the external load balancer routes, are automatically created.

As default in Microk8s you can use only services like NodePort and ClusterIP.

With your setup you can use NodePort, ClusterIP with Ingress or MetalLB. If you will use Ingress you have to remember to enable ingress and dns addons in Microk8s. It can be done by $ microk8s.enable dns ingress.

In short. Options you have are:

  1. Use NodePort
  2. Use Ingress Controller
  3. Set Endpoints in your YAMLs or use MetalLB and point Endpoints.

In Addition you can check similar case here and here.

Hope it helps.

Flores
  • 8,226
  • 5
  • 49
  • 81
PjoterS
  • 12,841
  • 1
  • 22
  • 54
0

You can also try simply adding to your Service (of the spec.type: LoadBalancer type) the so-called External IPs (see k8s docs and OCP docs, the latter incl. security considerations):

apiVersion: v1
kind: Service
metadata:
[..]
spec:
[..]
  externalIPs:
  - <external-ip>
  ports:
  - port: 8001 # port exposed to the k8s cluster
    targetPort: 8000 # containerized app's server port
  [..]
  type: LoadBalancer

Or using kubectl with the --external-ip flag:

kubectl expose [..] --type='LoadBalancer' --port=8001 --target-port=8000 [..] --external-ip='<external-ip>'

You also need to:

$ microk8s enable ingress
  • set up an Ingress, with its backend pointing to your service name and port (the one exposed to the cluster, not the outside world):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: <app-name>
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: <app-name>
            port:
              number: 8001

Then finally you can test the app with curl <external-ip>:80 or browse to http://<external-ip-or-hostname>.

Other options of routing traffic from the internet to the k8s cluster are rather limiting, requiring either using cloud providers network load balancers ("proper" spec.type: LoadBalancer) or an inconveniently large port number (spec.type: NodePort), likely to be blocked by your users corporate networks setups).

mirekphd
  • 4,799
  • 3
  • 38
  • 59