0

I have a program that launches a job using Job Controller. Due to some ops configuration errors the Pod is not able to pull the image from registry correctly.

I initially expect the Pod's status will go to "Failed", but turns out it stucks in "Pending" state forever. I initially thought I might be able to implement livenessProbe in the main container and set the timeout there, but since my init container is still waiting: ImagePullBackOff, the probe won't even start.

I wonder without inspection into the pod, is there a way to notify the Pod to fail when it fails to pull the image, e.g. setting a timeout or retry limit.

  • 1
    Possible duplicate of [Pods stuck in PodInitializing state indefinitely](https://stackoverflow.com/questions/53314770/pods-stuck-in-podinitializing-state-indefinitely) – cookiedough Mar 12 '19 at 20:46
  • @cookiedough The error in that question was due to init container failure. This is failing even before init container started, so I can't use livenessProbe. –  Mar 12 '19 at 20:51
  • If the init container is getting an ImagePullBackOff error that is also an init container failure. As you can see in the answers to the other question, the pod will be waiting until the init container succeeds based on the container restart policy. – cookiedough Mar 12 '19 at 20:55
  • I guess is there a way to catch this failure so that the Pod can result in a "Failed" state instead of Pending. I tried to add a livenessProbe to the main container, but it didn't work probably. https://kubernetes.io/docs/concepts/workloads/pods/init-containers/ says Use activeDeadlineSeconds on the Pod and livenessProbe on the Container to prevent Init Containers from failing forever. I'm not sure how that works. –  Mar 12 '19 at 22:08
  • How can it be re-created? For me it never goes to "Pending" `sukhoversha@sukhoversha:~/GCP$ kk get pods --watch NAME READY STATUS RESTARTS AGE webapp1 0/1 ImagePullBackOff 0 49s webapp1 0/1 ErrImagePull 0 1m webapp1 0/1 ImagePullBackOff 0 1m` – A_Suh Mar 13 '19 at 09:44
  • @A_Suh if you describe that pod, it will show status as Pending. –  Mar 13 '19 at 14:52
  • @kolorful can you pls share your job's manifest in order to reproduce – A_Suh Mar 14 '19 at 12:33

0 Answers0