"host not found in upstream ..." when using 'kubectl apply -f' but works in 'docker-compose up'

Question

I have four images in docker which are two services, a frontend and a reverseproxy.

When I was using docker-compose up, it works and the services were running.

But when I want to use kubectl apply -f to run in pod. There are [emerg] 1#1: host not found in upstream "backend-user:8080" in /etc/nginx/nginx.conf:11 in logs. I have no idea about this.

Here is my images:

bmjlearntocode/reverseproxy           latest
bmjlearntocode/udacity-frontend       local
bmjlearntocode/udacity-restapi-user   latest
bmjlearntocode/udacity-restapi-feed   latest

Here is the docker-compose.yaml

version: "3"
services:
  reverseproxy:
      image: bmjlearntocode/reverseproxy
      ports:
        - 8080:8080
      restart: always
      depends_on:
        - backend-user
        - backend-feed
  backend-user:
    image: bmjlearntocode/udacity-restapi-user
    volumes:
      - $HOME/.aws:/root/.aws
    environment:
      POSTGRESS_USERNAME: $POSTGRESS_USERNAME
      POSTGRESS_PASSWORD: $POSTGRESS_PASSWORD 
      POSTGRESS_DB: $POSTGRESS_DB 
      POSTGRESS_HOST: $POSTGRESS_HOST 
      AWS_REGION: $AWS_REGION 
      AWS_PROFILE: $AWS_PROFILE 
      AWS_BUCKET: $AWS_BUCKET
      JWT_SECRET: $JWT_SECRET
      URL: "http://localhost:8100"
  backend-feed:
    image: bmjlearntocode/udacity-restapi-feed
    volumes:
      - $HOME/.aws:/root/.aws
    environment:
      POSTGRESS_USERNAME: $POSTGRESS_USERNAME
      POSTGRESS_PASSWORD: $POSTGRESS_PASSWORD 
      POSTGRESS_DB: $POSTGRESS_DB 
      POSTGRESS_HOST: $POSTGRESS_HOST 
      AWS_REGION: $AWS_REGION 
      AWS_PROFILE: $AWS_PROFILE 
      AWS_BUCKET: $AWS_BUCKET
      JWT_SECRET: $JWT_SECRET
      URL: "http://localhost:8100"
  frontend:
    image: bmjlearntocode/udacity-frontend:local
    ports:
      - "8100:80"

Here is the nginx.conf

worker_processes 1;

events { worker_connections 1024; }
error_log /dev/stdout debug;

http {

    sendfile on;

    upstream user {
        server backend-user:8080;
    }

    upstream feed {
        server backend-feed:8080;
    }

    proxy_set_header   Host $host;
    proxy_set_header   X-Real-IP $remote_addr;
    proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header   X-Forwarded-Host $server_name;

    server {
        listen 8080;

        location /api/v0/feed {
            proxy_pass         http://feed;
        }
        location /api/v0/users {
            proxy_pass         http://user;
        }
    }

}

When I use docker-compose up, it works.

Here is the reverseproxy-deployment.ymal

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    service: reverseproxy
  name: reverseproxy
spec:
  replicas: 2
  template:
    metadata:
      labels:
        service: reverseproxy
    spec:
      containers:
      - image: bmjlearntocode/reverseproxy
        name: reverseproxy
        imagePullPolicy: Always          
        resources:
          requests:
            memory: "64Mi"
            cpu: "250m"
          limits:
            memory: "1024Mi"
            cpu: "500m"       
        ports:
        - containerPort: 8080
      restartPolicy: Always

I am not sure about why the nginx can not find the 'back-end' when the pod start, but can run in docker-compose up

score 6 · Accepted Answer · answered Feb 24 '20 at 22:34

6

I think its because backend-user only meaningful to docker network. When you define service in docker-compose file, you can access it with by its name. Obviously you cant access backend-user:8080 within kubernetes cluster because its undefined.

In kubernetes, you need Service resource for that kind of access. Also, you need Ingress resource or Service which its type nodeport for accesing apps from outside of kubernetes cluster(e.g from your browser).

you can check this resources. they are well written.

https://kubernetes.io/docs/tasks/access-application-cluster/service-access-application-cluster/ https://kubernetes.io/docs/tasks/access-application-cluster/ingress-minikube/

also you can put your nginx and back-end service in same pod and use localhost instead of hostname/servicename. so nginx can find upstream server.

answered Feb 24 '20 at 22:34

m303945

883
5
9

When I built the bmjlearntocode/reverseproxy, the Dockerfile of this image is ```COPY nginx.conf /etc/nginx/nginx.conf```, so I thought that I have defined the back-end service. But the reverseproxy did not know what the "back-end" is. So that is the problem? – bmjLearnToCODE Feb 25 '20 at 03:33
no, you doesnt define any kind of service. backend-user is domain name which is only available in docker network. Because its created automatically from docker-compose file. when nginx try to access backend-user it couldnt find it because backedn-user name cat not resolved by kubernetes network. – m303945 Feb 25 '20 at 07:39
You are right! There is no backend-user pod running, so the reverseproxy server cannot find that. Now I run the backend-user pod and its service to use 8080, and then I run the reverseproxy pod, it works. Thank you so much! – bmjLearnToCODE Feb 25 '20 at 17:01
How did you resolve this issue specifically? @bmjLearnToCODE I am having the same issue, did you rename your pod to backend-user or did you do anything extra?? – Elvis020 Sep 02 '22 at 10:40

score 0 · Answer 2 · answered Nov 13 '22 at 06:25

I think this was solved by using backend-user and backend-feed as service name instead of backend-user-svc and backend-feed-svc, OR modifying the nginx.conf and set the upstreams to user these two backend-user-svc and backend-feed-svc instead of backend-user and backend-feed

"host not found in upstream ..." when using 'kubectl apply -f' but works in 'docker-compose up'

2 Answers2