2

[xueke@master-01 admin]$ kubectl logs nginx-deployment-76bf4969df-999x8 Error from server (Forbidden): Forbidden (user=system:anonymous, verb=get, resource=nodes, subresource=proxy) ( pods/log nginx-deployment-76bf4969df-999x8)

[xueke@master-01 admin]$ kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://192.168.0.101:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: admin
  name: kubernetes
current-context: kubernetes
kind: Config
preferences: {}
users:
- name: admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED

I specified the admin user here How do I need to modify it?

user10830520
  • 21
  • 1
  • 1
  • 3

3 Answers3

3

The above error means your apiserver doesn't have the credentials (kubelet cert and key) to authenticate the kubelet's log/exec commands and hence the Forbidden error message.

You need to provide --kubelet-client-certificate=<path_to_cert> and --kubelet-client-key=<path_to_key> to your apiserver, this way apiserver authenticate the kubelet with the certficate and key pair.

For more information, have a look at:

https://kubernetes.io/docs/reference/access-authn-authz/kubelet-authn-authz/

Community
  • 1
  • 1
Prafull Ladha
  • 12,341
  • 2
  • 37
  • 58
0

In our case, the error stemmed from Azure services being downgraded because of a bug in DNS resolution, introduced in Ubuntu 18.04. See Azure status and the technical thread. I ran this command to set a fallback DNS address in the nodes:

az vmss list-instances -g <resourcegroup> -n vmss --query "[].id" --output tsv \
  | az vmss run-command invoke --scripts "echo FallbackDNS=168.63.129.16 >> /etc/systemd/resolved.conf; systemctl restart systemd-resolved.service" --command-id RunShellScript --ids @-
Johann8
  • 649
  • 7
  • 20
-1

That's an RBAC error. The user had no permissions to see logs. If you have a user with cluster-admin permissions you can fix this error with

kubectl create clusterrolebinding the-boss --user system:anonymous --clusterrole cluster-admin

Note: Not a good idea to give an anonymous user cluster-admin role. Will fix the issue though.

suren
  • 7,817
  • 1
  • 30
  • 51
  • RBAC shouldn't be used for providing anonymous users as cluster-admin access. This could be disastrous. See following issue for more info https://github.com/kubernetes/kubernetes/issues/55872: – Prafull Ladha Jan 08 '19 at 08:33
  • This is the worst "solution" I've ever seen on stackoverflow. People sometimes just mindlessly copy commands and that's a bad one. – monomo Oct 18 '21 at 21:17