docker nginx failed to make connection with web container

Question

My docker nginx container failed to connect with gunicorn container. docker compose logs looks like,

dj       | [2017-09-16 12:37:14 +0000] [22] [INFO] Starting gunicorn 19.7.1
dj       | [2017-09-16 12:37:14 +0000] [22] [DEBUG] Arbiter booted
dj       | [2017-09-16 12:37:14 +0000] [22] [INFO] Listening at: http://127.0.0.1:8000 (22)
dj       | [2017-09-16 12:37:14 +0000] [22] [INFO] Using worker: sync
dj       | [2017-09-16 12:37:14 +0000] [25] [INFO] Booting worker with pid: 25
dj       | [2017-09-16 12:37:14 +0000] [22] [DEBUG] 1 workers
ng       | 2017/09/16 12:37:22 [error] 8#8: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.20.0.1, server: localhost, request: "GET /api HTTP/1.1", upstream: "http://127.0.0.1:8000/api/v1", host: "localhost"
ng       | 172.20.0.1 - - [16/Sep/2017:12:37:22 +0000] "GET /api HTTP/1.1" 502 537 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/52.0.2743.116 Chrome/52.0.2743.116 Safari/537.36" "-"
ng       | 2017/09/16 12:37:31 [error] 8#8: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.20.0.1, server: localhost, request: "GET /admin HTTP/1.1", upstream: "http://127.0.0.1:8000/api/admin", host: "localhost"
ng       | 172.20.0.1 - - [16/Sep/2017:12:37:31 +0000] "GET /admin HTTP/1.1" 502 537 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/52.0.2743.116 Chrome/52.0.2743.116 Safari/537.36" "-"

If I make a curl request inside django container, it shows the relevant html text.

$ docker-compose exec api bash
root@e29e3756916c:/code# curl http://127.0.0.1:8000
root@e29e3756916c:/code# curl http://127.0.0.1:8000/api/v1
root@e29e3756916c:/code# curl -L http://127.0.0.1:8000/api/v1
{"_type":"document","_meta":{"url":"http://127.0.0.1:8000/api/v1/","title":"Backend APIs"},"auth":{"convert-token":{"create":{"_type":"link","url":"/api/v1/auth/convert-token/","action":"post","description":"Implements an endpoint to convert a provider token to an access token\n\nThe endpoint is used in the following flows:\n\n* Authorization code\n* Client credentials"}},"revoke-token":{"create":{"_type":"link","url":"/api/v1/auth/revoke-token/","action":"post","description":"Implements an endpoint to revoke access or refresh tokens"}},"sign_in":{"create":{"_type":"link","url":"/api/v1/auth/sign_in/","action":"post","encoding":"application/json","fields":[{"name":"username","required":true,"location":"form","schema":{"_type":"string","title":"Username","description":""}},{"name":"password","required":true,"location":"form","schema":{"_type":"string","title":"Password","description":""}}]}},"sign_up":{"create":{"_type":"link","url":"/api/v1/auth/sign_up/","action":"post","encoding":"application/json","fields":[{"name":"first_name","location":"form","schema":{"_type":"string","title":"First name","description":""}},{"name":"last_name","location":"form","schema":{"_type":"string","title":"Last name","description":""}},{"name":"email","location":"form","schema":{"_type":"string","title":"Email address","description":""}},{"name":"username","required":true,"location":"form","schema":{"_type":"string","title":"Username","description":"Required. 150 characters or fewer. Letters, digits and @/./+/-/_ only."}},{"name":"password","required":true,"location":"form","schema":{"_type":"string","title":"Password","description":""}},{"name":"is_active","location":"form","schema":{"_type":"boolean","title":"Active","description":"Designates whether this user should be treated as active. Unselect this instead of deleting accounts."}}]}},"token":{"create":{"_type":"link","url":"/api/v1/auth/token/","action":"post","description":"Implements an endpoint to provide access tokens\n\nThe endpoint is used in the following flows:\n\n* Authorization code\n* Password\n* Client credentials"}}}}root@e29e3756916c:/code# 
root@e29e3756916c:/code#

nginx.conf

server {
  listen       80;
  server_name  localhost;

  #charset koi8-r;
  #access_log  /var/log/nginx/host.access.log  main;

  #error_page  404              /404.html;

  # redirect server error pages to the static page /50x.html
  #
  error_page   500 502 503 504  /50x.html;
  location = /50x.html {
      root   /usr/share/nginx/html;
  }

  proxy_force_ranges      on;
  proxy_set_header        Host $host;
  proxy_set_header        X-Real-IP $remote_addr;
  proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header        X-Forwarded-Proto $http_x_forwarded_proto;

  location /admin {
    proxy_pass              http://127.0.0.1:8000/api/admin;
  }
  location /api {
    proxy_pass              http://127.0.0.1:8000/api/v1;
  }
  location /oauth {
    proxy_pass              http://127.0.0.1:8000/api/oauth;
  }
  location /static {
    proxy_pass              http://127.0.0.1:8000/static;
  }
  location / {
    root   /usr/share/nginx/html;
    index  index.html index.htm;
    try_files $uri $uri/ /index.html;
  }
}

Dockerfile for nginx container is,

FROM nginx:latest

#RUN apt-get update

ADD ./static /usr/share/nginx/html

ADD nginx.conf /etc/nginx/conf.d/default.conf

ENV NGINX_PORT 80

EXPOSE 80

CMD /bin/bash -c "nginx -g 'daemon off;'"

docker-compose.yaml

version: '2'  
services:  
  nginx:
    build: ./nginx
    container_name: ng
    ports:
      - "80:80"
    #volumes:
    #  - ./src:/src
    #  - ./config/nginx:/etc/nginx/conf.d
    depends_on:
      - api
    links:
      - api
    volumes_from:
      - api
  api:
    build: ./s2s_api
    container_name: dj
    command: bash ./wait_for_db.sh
    restart: always
    depends_on:
      - db
    ports:
      - "8000:8000"
    tty: true
    links: 
      - db:mysql

  db:
    image: mysql
    container_name: db
    command: mysqld --user=root --verbose
    volumes:
      - ./dbcreation.sql:/tmp/dbcreation.sql
      - ./import.sh:/tmp/import.sh
    ports:
      - "3306:3306"
    restart: always
    environment:
      MYSQL_DATABASE: "S2S"
      MYSQL_USER: "root"
      MYSQL_PASSWORD: "avi"
      MYSQL_ROOT_PASSWORD: "avi"
      MYSQL_ALLOW_EMPTY_PASSWORD: "yes"
      DB_PORT: 3306
      DB_HOST: db

wait_for_db.sh

#!/bin/sh
# Wait for database to get available

M_LOOPS="10"

#wait for mysql
i=0
# http://stackoverflow.com/a/19956266/4848859
while ! curl db:3306 >/dev/null 2>&1 < /dev/null; do
  i=`expr $i + 1`

  if [ $i -ge $M_LOOPS ]; then
    echo "$(date) - db:3306 still not reachable, giving up"
    exit 1
  fi

  echo "$(date) - waiting for db:3306..."
  sleep 3
done

echo "$(date) - db:3306 Reachable ! - Starting Daemon"
#start the daemon
#exec $START_CMD
python manage.py makemigrations
python manage.py migrate
gunicorn s2s_api.wsgi:application -b 127.0.0.1:8000 --log-level debug

Answer 1

You issue is obvious from the logs.

dj       | [2017-09-16 12:37:14 +0000] [22] [INFO] Listening at: http://127.0.0.1:8000 (22)
ng       | 2017/09/16 12:37:22 [error] 8#8: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.20.0.1, server: localhost, request: "GET /api HTTP/1.1", upstream: "http://127.0.0.1:8000/api/v1", host: "localhost"

So there are two issues and both of them at 127.0.0.1. In your dj container, you should be running gunicorn on 0.0.0.0:8000 because the request will be forwarded by the nginx container and it is external to django container

127.0.0.1 in each container points to the container loop back itself. Now when you proxy_pass in nginx to 127.0.0.1:8000, nginx is expecting something to be running in the same container at port 8000. So it doesn't work. You need to change it to name of the service you used in docker-compose. Assuming it was api, you should use

proxy_pass              http://api:8000/api/oauth;

Also on a side notem you shouldn't proxy pass static files to Gunicorn, they should be handled in nginx container itself