I am new to kubernetes and I finally realized how to launch the metrics-server as documented kubernetes-sigs/metrics-server. In case that someone else wonders you need to deploy on Master node and also have minimum one worker in the cluster.
So I get this error:
E0818 15:25:22.835094 1 manager.go:111] unable to fully collect metrics: [unable to fully scrape metrics from source kubelet_summary:<hostname-master>: unable to fetch metrics from Kubelet <hostname-master> (<hostname-master>): Get https://<hostname-master>:10250/stats/summary?only_cpu_and_memory=true: x509: certificate signed by unknown authority, unable to fully scrape metrics from source kubelet_summary:<hostname-worker>: unable to fetch metrics from Kubelet <hostname-worker> (<hostname-worker>): Get https://<hostname-worker>:10250/stats/summary?only_cpu_and_memory=true: x509: certificate signed by unknown authority]
I am using my own CAs (not self signed) and I have modified the components.yml file (sample):
args:
- --cert-dir=/tmp/metricsServerCas
- --secure-port=4443
- --kubelet-preferred-address-types=Hostname
I know that I can disable the tls by using this flag --kubelet-insecure-tls I have already tried it. I want to use my own CAs for extra security.
I have see other many relevant questions (few samples) e.g.:
x509 certificate signed by unknown authority- Kubernetes and kubectl unable to connect to server: x509: certificate signed by unknown authority
Although that I have applied chown already my $HOME/.kube/config still I see this error.
Where am I going wrong?
Update: On the worker I am creating a directory e.g. /tmp/ca and I add the ca file(s) in the directory.
I am not really good yet with the mounting points and I assume that I am doing something wrong. The default syntax of the images can be found here kubernetes-sigs/metrics-server/v0.3.7 (see components.yml file).
I tried to create a directory on my worker e.g. /tmp/ca and I modified the flag --cert-dir=/tmp/ca and mountPath: /tmp/ca
When I am deploying the file e.g.:
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml
I keep getting the error from the metrics-server-xxxx:
panic: open /tmp/client-ca-file805316981: read-only file system
Although I have given full access to the directory e.g.:
$ ls -la /tmp/ca
total 8
drwxr-xr-x. 2 user user 20 Aug 19 16:59 .
drwxrwxrwt. 18 root root 4096 Aug 19 17:34 ..
-rwxr-xr-x. 1 user user 1025 Aug 19 16:59 ca.crt
I am not sure where I am going wrong.
How is meant to be configured so someone can use non self signed certificates? I can see that most people are using non SSL which I would like to avoid.
Sample of my args in the image:
spec:
selector:
matchLabels:
k8s-app: metrics-server
template:
metadata:
name: metrics-server
labels:
k8s-app: metrics-server
spec:
serviceAccountName: metrics-server
volumes:
# mount in tmp so we can safely use from-scratch images and/or read-only containers
- name: tmp-dir
emptyDir: {}
containers:
- name: metrics-server
image: k8s.gcr.io/metrics-server/metrics-server:v0.3.7
imagePullPolicy: IfNotPresent
args:
- --cert-dir=/tmp/ca
- --secure-port=4443
- --kubelet-preferred-address-types=Hostname
ports:
- name: main-port
containerPort: 4443
protocol: TCP
securityContext:
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000
volumeMounts:
- name: tmp-dir
mountPath: /tmp/ca
nodeSelector:
kubernetes.io/os: linux
kubernetes.io/arch: "amd64"
Update 2: Adding curl command from Master to Worker including error output:
$ curl --cacert /etc/kubernetes/pki/ca.crt https://node_hostname:10250/stats/summary?only_cpu_and_memory=true
curl: (60) Peer's certificate issuer has been marked as not trusted by the user.
More details here: http://curl.haxx.se/docs/sslcerts.html
curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). If the default
bundle file isn't adequate, you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.
