kubectl through load balancer

Question

Flow:
kubernetes cluster with api server (https://192.168.0.10:6443) <-> load balancer (10.10.0.2) <-> laptop.

Idea:
From my laptop I would like to run kubectl pointing to load balancer where reveres proxy will redirect me to api server.

Steps:
- I changed server ip in kubeconfig (on my laptop) file to LB's IP:
was https://192.168.0.10:6443 is http://10.10.0.2:8080/
- I configured nginx like that:

server {
    listen 8080 default_server;
    listen [::]:8080 default_server;
    server_name _;
    location / {
            proxy_pass https://192.168.0.10:6443;
    }
}

Now running for example kubectl get nodes I expected to get list of nodes but it won't work:

error: You must be logged in to the server (Unauthorized)

$ curl http://10.10.0.2:8080/
{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "Unauthorized",
  "reason": "Unauthorized",
  "code": 401

If I add to nginx config:

ssl on;
ssl_certificate /root/certs/admin-k-master-1.pem;
ssl_certificate_key /root/certs/admin-k-master-1-key.pem;

and change in kubeconfig file server IP to https://10.10.0.2:8080/

$ kubect get nodes
Unable to connect to the server: x509: certificate is valid for 192.168.0.10 not 10.10.0.2

There is similar topic but it's not related to kubectl.

How can I achieve that? or what I am doing wrong.

Answer 1

One of the solutions is to add to nginx config file:

proxy_pass 127.0.0.1:8001;

and in load-balancer instance run:

kubectl proxy.

Then it works.

Answer 2

From my laptop I would like to run kubectl pointing to load balancer where reveres proxy will redirect me to api server.

• This is how we configured such a loadbalancer for external access. Mind that you need some kind of ssl certificate for this to work with ssl all-accross. We do have proper certificate for my-domain.com (namely kubernetes.my-domain.com) in place but you might have to experiment with self signed depending on your actual ip/dns names. Also note that kubernetes is name of our cluster, so if your name is different you will have to update upstream reference as well.

  upstream kube {
     server kubernetes:443;
  }
  
  server {
     listen 80;
     server_name kubernetes.my-domain.com;
     root /nowhere;
     rewrite ^ https://kubernetes.my-domain.com$request_uri permanent;
   }
  
  server {
     listen 443 ssl;
     server_name kubernetes.my-domain.com;
  
     ssl_certificate      /etc/nginx/ssl/kubernetes.my-domain.com.crt;
     ssl_certificate_key  /etc/nginx/ssl/kubernetes.my-domain.com.key;
  
     location / {
       proxy_set_header        Host $host:$server_port;
       proxy_set_header        X-Real-IP $remote_addr;
       proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
       proxy_set_header        X-Forwarded-Proto $scheme;
       proxy_redirect http:// https://;
       proxy_pass              https://kube;
       # Required for new HTTP-based CLI
       proxy_http_version 1.1;
       proxy_request_buffering off;
       proxy_buffering off; # Required for HTTP-based CLI to work over SSL
     }
  }
  

Answer 3

Put 10.10.0.2 into 'alt_name' part of openssl.cnf and rebuild your apiserver key pair, then apiserver will treat queries to 10.10.0.2 as valid.