5

I am new for pod health check with Readiness and Liveness. Recently I am working on Readiness. The scenario is as following:

The pod is a RestAPI service, it needs to connect to Database and store information in DB. So if RestAPI service wants to offer service, it needs to make sure database connection is successfully.

Si in our pod Readiness logic implementation, we use HTTP-Get and check if DB connection is connected, if it is okay, then HTTP-Get returns Ok, otherwise Readiness will be failed.

Not sure if the above logic is reasonable? Or is there any other approach for this logic processing?

Apart from Readiness, how about Liveness? Do I need to check DB connection in order to check Liveness is okay?

Any idea and suggestion are appreciated

Joe
  • 623
  • 7
  • 16

1 Answers1

3

readiness and liveness is mostly for service you are running inside container, there could be a scenario where your DB is up but there is issue with the application at that time also your readiness will be Up as DB is running, in ideal scenario if application not working it should stop accepting traffic.

i would recommend using the Init container or Life cycle hook for checking the condition of the Database first if it's up process will move ahead and your application or deployment will come into the picture.

If the application works well your readiness and liveness will HTTP-OK and the service start accepting traffic.

init container example

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
  labels:
    app: myapp
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', 'echo The app is running! && sleep 3600']
  initContainers:
  - name: init-myservice
    image: busybox
    command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;']
  - name: init-mydb
    image: busybox
    command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;']

Extra Notes

There is actually no need to check the DB readiness at all.

As your application will be trying to connect with the Database so if DB is not UP your application won't respond HTTP-OK so your application won't start, and readiness keep failing for application.

As soon as your Database comes up your application will create a successful connection with DB and it will give 200 responses and readiness will mark POD ready.

there is no extra requirement to setup the readiness for Db and based on that start POD.

Harsh Manvar
  • 27,020
  • 6
  • 48
  • 102
  • In your examples, we need a initContainer such as as busy box. Suppose we have no proper init container image, for example, out dependency is a CustomResouce in the K8s, we need to check if this CR is available, if not available, then our service is not ready for offering service. How about this scenario? Could we check CR status in our Readiness? – Joe Nov 22 '21 at 17:16
  • if you just want to check the CRD exist or not you can use the simple `kubectl` run `kubectl` command line in `init container` or else write down custom docker image with code in python or so which will use the K8s-client and check if CRD exist or not. – Harsh Manvar Nov 22 '21 at 17:27
  • as your container will be running on cluster that wont be much issue authenticating the K8s client and you can simply check status. ref :https://stackoverflow.com/a/68710305/5525824 – Harsh Manvar Nov 22 '21 at 17:27
  • Thanks prompt reply. The above approach is another one. They are both using init container approach. The 3rd, is it possible that in our HTTP-Get we check if CR is available ? That means we implement it in our Readiness code. – Joe Nov 22 '21 at 17:40
  • Yes but again not sure why you want to check crd? If crd not exist init container will fail, readiness will also. So better let application fail scenario suggested in extra note – Harsh Manvar Nov 22 '21 at 19:12
  • What is the dB connection breaks after the app was ready? Init container won't catch that. – The Fool Sep 10 '22 at 07:22
  • If that the case thn ideally it should catch by app it self liveness or readiness, however i think that exception should be handle at code level again depends on usecase can be handled it. – Harsh Manvar Sep 10 '22 at 07:48