Nginx upstream server values when serving an API using docker-compose and kubernetes

Question

I'm trying to use docker-compose and kubernetes as two different solutions to setup a Django API served by Gunicorn (as the web server) and Nginx (as the reverse proxy). Here are the key files:

default.tmpl (nginx) - this is converted to default.conf when the environment variable is filled in:

upstream api {
    server ${UPSTREAM_SERVER};
}

server {
    listen 80;

    location / {
        proxy_pass http://api;
    }

    location /staticfiles {
        alias /app/static/;
    }
}

docker-compose.yaml:

version: '3'
services:
  api-gunicorn:
    build: ./api
    command: gunicorn --bind=0.0.0.0:8000 api.wsgi:application
    volumes:
      - ./api:/app

  api-proxy:
    build: ./api-proxy
    command: /bin/bash -c "envsubst < /etc/nginx/conf.d/default.tmpl > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
    environment:
      - UPSTREAM_SERVER=api-gunicorn:8000
    ports:
      - 80:80
    volumes:
      - ./api/static:/app/static
    depends_on:
      - api-gunicorn

api-deployment.yaml (kubernetes):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: release-name-myapp-api-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: myapp-api-proxy
  template:
    metadata:
      labels:
        app.kubernetes.io/name: myapp-api-proxy
    spec:
      containers:
        - name: myapp-api-gunicorn
          image: "helm-django_api-gunicorn:latest"
          imagePullPolicy: Never
          command:
            - "/bin/bash"
          args:
            - "-c"
            - "gunicorn --bind=0.0.0.0:8000 api.wsgi:application"
        - name: myapp-api-proxy
          image: "helm-django_api-proxy:latest"
          imagePullPolicy: Never
          command:
            - "/bin/bash"
          args:
            - "-c"
            - "envsubst < /etc/nginx/conf.d/default.tmpl > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
          env:
            - name: UPSTREAM_SERVER
              value: 127.0.0.1:8000
          volumeMounts:
            - mountPath: /app/static
              name: api-static-assets-on-host-mount
      volumes:
        - name: api-static-assets-on-host-mount
          hostPath:
            path: /Users/jonathan.metz/repos/personal/code-demos/kubernetes-demo/helm-django/api/static

My question involves the UPSTREAM_SERVER environment variable.

For docker-compose.yaml, the following values have worked for me:

Setting it to the name of the gunicorn service and the port it's running on (in this case api-gunicorn:8000). This is the best way to do it (and how I've done it in the docker-compose file above) because I don't need to expose the 8000 port to the host machine.
Setting it to MY_IP_ADDRESS:8000 as described in this SO post. This method requires me to expose the 8000 port, which is not ideal.

For api-deployment.yaml, only the following value has worked for me:

Setting it to localhost:8000. Inside of a pod, all containers can communicate using localhost.

Are there any other values for UPSTREAM_SERVER that work here, especially in the kubernetes file? I feel like I should be able to point to the container's name and that should work.

score 1 · Accepted Answer · answered Apr 07 '20 at 13:20

You could create a service to target container myapp-api-gunicorn but this will expose it outside of the pod:

apiVersion: v1
kind: Service
metadata:
  name: api-gunicorn-service
spec:
  selector:
    app.kubernetes.io/name: myapp-api-proxy
  ports:
    - protocol: TCP
      port: 8000
      targetPort: 8000

You might also use hostname and subdomain fields inside a pod to take advantage of FQDN.

Currently when a pod is created, its hostname is the Pod’s metadata.name value.

The Pod spec has an optional hostname field, which can be used to specify the Pod’s hostname. When specified, it takes precedence over the Pod’s name to be the hostname of the pod. For example, given a Pod with hostname set to “my-host”, the Pod will have its hostname set to “my-host”.

The Pod spec also has an optional subdomain field which can be used to specify its subdomain. For example, a Pod with hostname set to “foo”, and subdomain set to “bar”, in namespace “my-namespace”, will have the fully qualified domain name (FQDN) “foo.bar.my-namespace.svc.cluster-domain.example”.

Also here is a nice article from Mirantis which talks about exposing multiple containers in a pod

Nginx upstream server values when serving an API using docker-compose and kubernetes

1 Answers1