17

I have searched some but could not get much as I am new to k8s. My pods are evicted and I get message as:

"Status:        Failed
Reason:         Evicted
Message:        The node was low on resource: nodefs."

Any help how can I figure out what is going on?

jdhurst
  • 4,365
  • 1
  • 20
  • 21
AhmFM
  • 1,552
  • 3
  • 23
  • 53

1 Answers1

20

Run kubectl describe pod <pod name> and look for the node name of this pod. Followed by kubectl describe node <node-name> that will show what type of resource cap the node is hitting under Conditions: section.

From my experience this happens when the host node runs out of disk space.

ZPrime
  • 393
  • 4
  • 8
  • They look ok. Conditions: Type Status LastHeartbeatTime LastTransitionTime Reason Message ---- ------ ----------------- ------------------ ------ ------- OutOfDisk False Mon, 26 Nov 2018 11:49:32 -0800 Mon, 05 Nov 2018 16:09:45 -0800 KubeletHasSufficientDisk kubelet has sufficient disk space available – AhmFM Nov 26 '18 at 19:57
  • but you suggestion did help me get to cause. Thanks!. however not sure what else could I do to find the reason? will it be just to delete pods now as last resort? – AhmFM Nov 26 '18 at 21:12
  • It's hard to say then. You can try starting the pod again and monitor the describe command `watch kubectl describe node <>` and see if the condition values change. Also after starting the pod you should run `kubectl logs -f` and see if there are anything helpful on the log output. – ZPrime Nov 26 '18 at 21:18