0

I deployed WAS to Kubernetes(version 1.16). I used all three types of probes.

The Liveness probe is set to check if the WAS process is running and if all open ports are listening. The Readiness probe calls the healthcheck api of WAS via http get. The Startup probe uses the same logic as the Liveness probe, but has an additional task to init the healthcheck api. This means that if the Startup probe is not executed, the healthcheck api will not be enabled, and the readiness probe will always fail.

My guess is that

  1. if the Startup probe fails repeatedly beyond a threshold, the container will be restarted.
  2. if the startup probe ran normally, the readiness probe shouldn't fail. (Note that the startup probe also fails if the h.c. api enabling fails, so there is no case where the readiness probe fails even though the startup probe succeeds).

In conclusion, there should not be a situation where the readiness probe fails, because the only cases where it does are when the container is restarted or the readiness probe succeeds. In addition, the startup probe has a threshold of 36 times and a period of 5 seconds, so the liveness/readiness probe should not run for 180 seconds. However, there are cases where the readiness probe fails before 3 minutes.

This leads me to believe that the behavior of the startup probe is overridden and the liveness/readiness probe is executed.

According to the kubernetes docs, the startup probe is a probe to ensure that the liveness/readiness probe runs at the right time. The problem is that if this probe is ignored, timing in absolute time via initialDelaySeconds is not as good as timing in absolute time.

First of all, I'm wondering if the problem I'm guessing actually happens. I also don't know how to verify this. Even in k8s events, I could only see the readiness probe failed event, not the success/failure of the startup probe. Maybe I misunderstood how the startup probe works. I hope someone can provide a proper solution.

Below is the configuration of the probe I wrote.

livenessProbe:
  exec:
    command:
    - liveness 
  initialDelaySeconds: 10
readinessProbe:
  exec:
    command:
    - readiness
  initialDelaySeconds: 10
startupProbe:
  exec:
    command:
    - liveness
    - -startup
  failureThreshold: 36
  periodSeconds: 5

Analyze logs with commands like kubectl describe and kubectl logs Checking k8s events, etc.

Eddy Kim
  • 33
  • 2

1 Answers1

2

The startup probe is not enabled by default for 1.16 version https://v1-22.docs.kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/. To use this probe, you have to enable feature gate.

k0chan
  • 681
  • 1
  • 6
  • 14