k8s user container is getting shut down by envoy/istio sigterm without completing process

Question

Envoy sending TERM signal and closing my application before the process completion. Increasing Drain Duration preventing the pod from going down after the completion.(its waiting for the grace duration to be over and not allowing any new requests.

Expectation:

1. After getting Envoy Sigterm, istio should wait terminationDrainDuration seconds for my process in user container is completed. if the process not completed in the grace duration, It should terminate(kill) all the remaining process.
2. If my process in user container got completed before the grace period is over, istio should not wait for that and immediately send kill signal.

Is there any way to check the completion of process in user container and send kill signal if there is no process alive?

Process running in user container is Gunicorn process.

Answer 1

It's not clear from your post if you're running Knative, or if this is a general Istio question. (It has Knative tags, but doesn't mention Knative.) I'm also assuming you're running Istio on Kubernetes. This isn't a requirement for Istio, but is a common configuration.

Based on your description, it sounds like you have the lifecycles of the two components slightly confused. Istio isn't responsible for the lifecycle of your application, kubelet does this. You might need to set terminationGracePeriodSeconds to give your application an early notice before it is due down my kubelet.