12

readinessProbe: Indicates whether the container is ready to respond to requests. If the readiness probe fails, the endpoints controller removes the Pod's IP address from the endpoints of all Services that match the Pod. The default state of readiness before the initial delay is Failure. If a Container does not provide a readiness probe, the default state is Success

If the readiness probe fails (and the Pod's IP address is removed from end point), what happens next? Will the Pod's readiness probe conditions be checked again? Will it check again after initial delay? Is there any chance the Pod's IP address is added to the end point again (if the Pod self healed after readiness probe fails)? Will that Pod receive traffic again incase if it's healed?

Raedwald
  • 46,613
  • 43
  • 151
  • 237
User5678
  • 191
  • 1
  • 8
  • Are you asking about a readiness probe or a likeness probe? – Raedwald Jun 14 '21 at 06:19
  • hi @Raedwald i asked about readiness probe not liveness probe – User5678 Jun 14 '21 at 06:21
  • Readiness state is completely independent from liveliness state. If once not ready the pod was never selected again then the readiness probe would be the liveness probe except you leave garbage running instead of restarting it. You might want to explicitly remove a pod from service, for example while it is running some expensive computations/batch job maybe, and when it finished you may want to get it back online. – GACy20 Jun 14 '21 at 10:24
  • @GACy20 yes it is constantly checked in period time interval even if it became garbage. so garbage also constantly evaluated right! – User5678 Jun 14 '21 at 11:22

2 Answers2

8

will the pod's readiness prob conditions checked again?

yes, the condition will be checked again depends on the threshold you have set.

On each periodSecondsconfiguration readiness will be checked for POD.

will it check again after the initial delay?

It will check after the Initial delay only. Initial delay comes into the picture when POD is initializing or starting. readiness check will wait for configured time and after that time it will start checking the readiness of POD on each time interval suppose every 5 second or 10 seconds depends on the configuration of the periodSeconds.

are there any chance pod's ip address added to the end point again(if pod self healed after readiness probe fails)?

Yes if get auto healed mean, successThreshold set to 1 time if POD give 200 one time it will marked as heal and running pod is this case POD will get the traffic again.

will the pod ever receive traffic again incase if it's healed?

Yes

For example :

readinessProbe:
            httpGet:
              path: /k8/readiness
              port: 9595
            initialDelaySeconds: 25
            periodSeconds: 8
            timeoutSeconds: 10
            successThreshold: 1
            failureThreshold: 30
        livenessProbe:
            httpGet:
              path: /k8/liveness
              port: 9595
            initialDelaySeconds: 30
            periodSeconds: 8
            timeoutSeconds: 10
            successThreshold: 1
            failureThreshold: 30

The readiness and liveness probe will check the status on HTTP endpoint as mentioned in configuration.

initialDelaySeconds : it will only come into the picture when your POD initializing or starting again due to restart or anything. So when POD is starting readiness wont check service status till 30 second.

After 30 seconds it will try to check the status on the endpoint. If successful POD will be in a ready state to handle the traffic or else it will try for another time periodSeconds so after 8 seconds it will try again if we will 200 response POD will be Ready or else will try after 8 seconds.

timeoutSeconds : single hop or request will wait the amount of time to get response from service or else mark as failed check.

failureThreshold : Maximum number of failed check after this POD will get started or changed to Not Ready state based on configuration liveness or readiness.

successThreshold : Successful threshold means if single request get success response from service POD status gets changed to Ready.

If continuous 30 failureThreshold occur then only POD will get marked as Not ready if in between single successThreshold occur POD will get mark as Ready same for liveness.

Note : Above example is just for reference can not be useful in an actual production scenario.

Read more at : https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/

Harsh Manvar
  • 27,020
  • 6
  • 48
  • 102
  • 1
    Hi @Harsh manvar i still need some clarity on that part. i've edited my question. can you answer for it? – User5678 Jun 14 '21 at 05:08
  • Sure, please give me min. – Harsh Manvar Jun 14 '21 at 05:51
  • 1
    yes it will check the liveness and readiness each time based on interval set. and pod can come back to healthy and ready state after failing liveness and readiness check one time. – Harsh Manvar Jun 14 '21 at 05:53
  • 1
    take this example of configuration : `initialDelaySeconds: 25 periodSeconds: 8 timeoutSeconds: 10 successThreshold: 1 failureThreshold: 30` initial delay second mean after 25 second readiness will check pod can handle request or not. one ready after every 8 second readiness check will be there. – Harsh Manvar Jun 14 '21 at 05:58
  • 1
    will wait 10 second timeout for response to read form service. if we will get 1 time 200 response from service it will mark as ready due to `successThreshold`. if in serial line readiness check fail 30 time and get 404 or 400 status will get changed to `Not ready` and POD get `restarted` and again check will start but after 25 seconds as a delay there if POD comes up and start working it will start accepting traffic. – Harsh Manvar Jun 14 '21 at 06:00
  • 1
    if your service taking less time to start reduce the `initialDelaySeconds` to 2-3 second also but if it's heavy lifting and service taking time around 30-40 second adjust it that way. also set the `periodSeconds` mean one how much interval you want to run the readiness check every 8second. – Harsh Manvar Jun 14 '21 at 06:03
  • 1
    Hi @Harsh manvar so in readiness probe once pod became ready to unready. in which time pod will be again checked after 25(initial delay) seconds or 8 seconds(period seconds)? – User5678 Jun 14 '21 at 06:17
  • 1
    one pod is starting it will wait for 25 second initial delay, it will check the readiness once if ready pod will be ready or else after 8 second it will check again on endpoint. – Harsh Manvar Jun 14 '21 at 07:20
  • 1
    hi @Harsh Manvar i saw your edited answer i have an doupt what is benefit on using liveness probs and readiness probs together? you have used same parameters for readiness and liveness probs i couldn''t understand what benefits which provides. can you please explain? same scenario i seen in many examples. – User5678 Jun 14 '21 at 13:28
  • 1
    that's why i have mentioned in Note do not use it is for ref only, however consider readiness probe when you want to add your pod behind load balancer and it can handle the request. while liveness can be useful to check the application status. it might be possible liveness endpoint checking all Db connections and other stuff in background when we hit the endpoint. – Harsh Manvar Jun 14 '21 at 13:32
  • 1
    depends on the scenario you can use as same also no issue in it but liveness is the one who restarts the pod if your application is dead inside the POD. read this good article : https://cloud.google.com/blog/products/containers-kubernetes/kubernetes-best-practices-setting-up-health-checks-with-readiness-and-liveness-probes – Harsh Manvar Jun 14 '21 at 13:35
  • I saw many examples in internet using readiness and liveness probs in same container with same parameters. I don't understand the purpose why they're using two probs on same container that's why asked – User5678 Jun 14 '21 at 13:44
  • 1
    clear thing is that, readiness donot restart the pod but liveness restart the pod. if your application fails readiness will change status and it will be out of service but if liveness failing it will restrt the the POD or container. – Harsh Manvar Jun 14 '21 at 14:15
4

It's checked again after the same periodSeconds delay as usual and then when it passes successThreshold times in a row it will be considered Ready again with all the normal behaviors that entails.

coderanger
  • 52,400
  • 4
  • 52
  • 75
  • hi @coderanger do you say it only check again after the periodseconds not consider initial delay? – User5678 Jun 14 '21 at 04:20
  • No, that is only used when the container is starting. It was how we did startup delays before startup probes were added. – coderanger Jun 14 '21 at 04:22
  • hi @codranger giving up in case of reading probe the pod will be marked unready. my question is Will the pod's readiness prob conditions checked again after giving up? will that pod become ready? – User5678 Jun 14 '21 at 05:15
  • 1
    As my answer says, yes after `successThreshold` successful probes the container will be marked as Ready again. – coderanger Jun 14 '21 at 05:17