3

Assume I have an app A container and another container called resources-preparation which will try to create DB tables and etc in order to bootstrap app A.

App A container and resources-preparation container are living in different pods. How can I bring up App A container after resources-preparation container completes.

PS: resources-preparation container is not a service at all. So I may not be able to use the waitfor image to detect the resources-preparation container completes.

injoy
  • 3,993
  • 10
  • 40
  • 69
  • 1
    Why isn't `initContainers` a solution here? The problem is that you'll need your app `A` to be aware of it's underlying infrastructure, that's not a very good idea. Can't you manage that a deployment time? (first deploy `resources-preparation`, wait for it to finish and then deploy app `A`?) – Pierre B. Jul 26 '19 at 07:08

4 Answers4

3

It seems there is a kind of architectural inconsistency: the existing application architecture does not fit Kubernetes paradigm well:

  • The pod A is tied to the pod resources-preparation, so has to wait for its successful completion, whereas k8s assumes independent or loosely coupled microservices.
  • Being tightly dependent, containers A and resources-preparation are put into different pods whereas assisting applications should be placed into the same container with the primary one. See the Discussion Communicate Between Containers in the Same Pod.
  • The pod A is dependent on an external database whereas in k8s microservices should work with their own database or replica to keep independency.
  • The pods A and resources-preparation should communicate via k8s API. That means the pod A should fetch information about the resources-preparation completion from the kube-apiserver.

The listed principles cause extra overhead but this is the price you pay for the redundancy Kubernetes relies on.

Possible approaches to the problem:

  1. Redesign or modify the application and backend database accordingly with the k8s principles, decompose them into a set of loosely coupled microservices. As a supposal:

    • a) let the app A start with its DB replica independently;
    • b) in parallel let the resources-preparation to start and create tables in its own replica;
    • c) then add the new tables to the existing Replication or create a new Replication. In this approach the pod A does not have to wait for the pod resources-preparation. The DB replication will be waiting instead. That way the dependency will be moved off the k8s level to the upper layer.

    Unfortunately, adaptation of existing applications to k8s could be challenging and often requires re-developing the application from scratch. It is the time- and resource-consuming task.

    A good whitepaper is available here: Principles of container-based application design.

  2. Since the resources-preparation is an assisting container for the A, put both containers into the same pod. That way the sample code from the Init Containers concept will do exactly the container A needs. What's important for the container A awaiting for the resources-preparation completion is that:

    • Init containers always run to completion.
    • Each init container must complete successfully before the next one starts.

  3. If you can not join both containers into the same pod for some reason, as a workaround the application components could be put into a "wrapper" that helps them to pretend behaving as loosely coupled microservices. This wrapper should be implemented below the pod level to be transparent for Kubernetes: around the container or application. In a simple case you might launch the application A from within a shell script with the until loop. The script should fetch the status of the resources-preparation pod running in a StatefulSet via the kube-apiserver to make decision if the application A may start or not.

    A REST API request could be used for that (see the answer Kubernetes API server, serving pod logs).

    A way to authenticate on the kube-apiserver should be provided for the API request to work. The theory and practical examples are here:

    Access Clusters Using the Kubernetes API

    cURLing the Kubernetes API server

mebius99
  • 2,495
  • 1
  • 5
  • 9
1

App A container and resources-preparation container are living in different pods.

There is dependency among pods with conditions in your use-case.

I would try to use wait command till a specific condition is true

kubectl wait --for=condition=complete job/myJob

After than you can rollout your deployment

for further detail kubectl-wait

Suresh Vishnoi
  • 17,341
  • 8
  • 47
  • 55
0

did you consider to look at the initContainers?

In your case, A container and resources-preparation container are living in different "PODS" so never tried that but take a look at the https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-initialization/

Salman Memon
  • 31
  • 3
  • Yes, I used initContainers in other scenarios. However, this problem here is to detect the other completes execution. Is there an easy way to accomplish this with initContainer? – injoy Jul 26 '19 at 03:23
  • nope, it's interesting, looking forward to seeing the solution for this. – Salman Memon Jul 26 '19 at 04:32
0

you need to use initContainer. refer the below code. in the belwo example The container first waits for myservice, and second waits for mydb. Once both init containers complete, the Pod runs myapp-container

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
  labels:
    app: myapp
spec:
  containers:
  - name: myapp-container
    image: busybox:1.28
    command: ['sh', '-c', 'echo The app is running! && sleep 3600']
  initContainers:
  - name: init-myservice
    image: busybox:1.28
    command: ['sh', '-c', 'until nslookup myservice; do echo waiting for myservice; sleep 2; done;']
  - name: init-mydb
    image: busybox:1.28
    command: ['sh', '-c', 'until nslookup mydb; do echo waiting for mydb; sleep 2; done;']
P Ekambaram
  • 15,499
  • 7
  • 34
  • 59
  • FYI there's a better way of waiting for service to be live: `nc -z -w 2 host:port`. Try and google `nc -z -w 2 GitHub helm charts` if you're interested. What it does is waits for TCP connection to respond, that is at least one `endpoint` must exist on the `service`, which in turn means that at least one `pod` from this `service` is in `ready` state. – Egor Stambakio Jul 26 '19 at 15:34
  • Guys, if you have read the initial question carefully, you'd notice that the container `A` is waiting for the successful **completion** of the container `resources-preparation` in another pod. Hence: 1) the offered configuration with `initContainers` is not applicable; 2) the `resources-preparation` container is **not a service** so waiting for the alive service with `nslookup` or `nc` is not applicable as well. – mebius99 Jul 29 '19 at 08:18
  • the sample code is a reference. instead of waiting for service, check for container endpoint – P Ekambaram Jul 29 '19 at 11:34
  • I think I need to wait for the `resources-preparation` container completes execution. So using the `nslookup` or `nc` command might not guarantee this. – injoy Jul 29 '19 at 17:23