How do you deploy properly to AWS with Elastic beanstalk and Dockerrun.aws.json?

Question

I currently have a docker image that works locally and is privately hosted on hub.docker.com.

Inside the container I have rails, puma, nginx. Elastic beanstalk is able to pull the image successfully from docker hub but fails to do anything afterwards.

AWS has nginx and is returning me this error. Anyone able to point out what I am doing wrong?

AWS Error Log

-------------------------------------
/var/log/nginx/error.log
-------------------------------------
2014/12/27 08:48:34 [emerg] 3161#0: no host in upstream ":80" in           /etc/nginx/conf.d/elasticbeanstalk-nginx-docker-upstream.conf:2

More AWS Error logs

nginx: [emerg] no host in upstream ":80" in /etc/nginx/conf.d/elasticbeanstalk-nginx-    docker-upstream.conf:2
nginx: configuration file /etc/nginx/nginx.conf test failed
Failed to start nginx, abort deployment (Executor::NonZeroExitStatus)
    at /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/executor-1.0/lib/executor/exec.rb:81:in `sh'
    from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/executor-1.0/lib/executor.rb:15:in `sh'
    from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/beanstalk-core-1.0/lib/elasticbeanstalk/executable.rb:63:in `execute!'
    from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/beanstalk-core-1.0/lib/elasticbeanstalk/hook-directory-executor.rb:29:in `block (2 levels) in run!'
    from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/beanstalk-core-1.0/lib/elasticbeanstalk/activity.rb:169:in `call'
    from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/beanstalk-core-1.0/lib/elasticbeanstalk/activity.rb:169:in `exec'
    from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/beanstalk-core-1.0/lib/elasticbeanstalk/activity.rb:126:in `timeout_exec'
    from /opt/elasticbeanstalk/lib/ruby/lib/ruby/gems/2.1.0/gems/beanstalk-core-1.0/lib/elasticbeanstalk/activity.rb:110:in `block

The src file of the error

upstream docker {
        server :80;
        keepalive 256;
}

Here is the files I have.

Dockerfile

FROM ruby:2.1.5

#################################
# native libs
#################################

RUN apt-get update -qq
RUN apt-get install -qq -y build-essential
RUN apt-get install -qq -y libpq-dev
RUN apt-get install -qq -y nodejs
RUN apt-get install -qq -y npm
RUN apt-get install -qq -y nginx

# Clean up APT when done.
RUN apt-get clean && rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*

#################################
# Install Nginx.
#################################

RUN echo "\ndaemon off;" >> /etc/nginx/nginx.conf
RUN chown -R www-data:www-data /var/lib/nginx
ADD config/nginx.conf /etc/nginx/sites-enabled/default

EXPOSE 80

#################################
# Symlinking Nodejs for ubuntu
#   -- http://stackoverflow.com/questions/26320901/cannot-install-nodejs-usr-bin-env-node-no-such-file-or-directory
#################################
RUN ln -s /usr/bin/nodejs /usr/bin/node

#################################
# NPM install globals
#################################

RUN npm install bower -g

#################################
# Rails
#################################

RUN mkdir /app
WORKDIR /app
ADD . /app

ENV RAILS_ENV production
ENV SECRET_KEY_BASE test123

RUN bundle install --without development test
RUN bundle exec rake bower:install
RUN bundle exec rake assets:precompile

CMD foreman start -f Procfile

Dockerrun.aws.json

{
  "AWSEBDockerrunVersion": "1",
  "Authentication": {
    "Bucket": "aws-bucket",
    "Key": ".dockercfg"
  },
  "Image": {
    "Name": "ericraio/my-image",
    "Update": "true"
  },
  "Ports": [
    {
      "ContainerPort": "80"
    }
  ],
  "Logging": "/var/log/nginx"
}

NGINX

upstream rails_app {
  server unix:///app/tmp/sockets/puma.sock fail_timeout=0;
}

server {
  # listen 80 deferred;
  # server_name domain.tld www.domain.tld;
  root /app/public;

  try_files $uri/index.html $uri @rails_app;

  location @rails_app {
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_redirect off;
    proxy_pass http://rails_app;
  }

  error_page 500 504 /500.html;
  error_page 502 /502.html;
  error_page 503 /503.html;

  client_max_body_size 4G;
  keepalive_timeout 10;
}

score 7 · Accepted Answer · answered Dec 29 '14 at 06:14

7

The issue is that on EC2 the container was failing to run. Amazon uses this command to build an IP address and setting an environment variable.

EB_CONFIG_NGINX_UPSTREAM_IP=$(docker inspect `cat $EB_CONFIG_DOCKER_STAGING_APP_FILE` | jq -r .[0].NetworkSettings.IPAddress)

Then amazon uses this line to build the IP address.

EB_CONFIG_NGINX_UPSTREAM_PORT=`cat $EB_CONFIG_DOCKER_STAGING_PORT_FILE`

I had my port 80 exposed but because my container would fail to run I did not have a host. Which is why you get this error.

no host in upstream ":80"

answered Dec 29 '14 at 06:14

ericraio

1,469
14
35

But why does container fail. I'm deploying a java app inside Docker and getting same error on :9000. – Chris Fellows Mar 24 '15 at 22:18
2

@ChrisFellows if your container fails to run then amazon can not build an ip address. Amazon really needs to output an error message. Try building and running the container locally to see what your output is. – ericraio Mar 25 '15 at 22:55
Well running locally is fine. Also doing a docker ps -a shows me that the docker container is up and running. But if I try to curl 127.0.0.1:9000 (that's what is EXPOSE'd in the Dockerfile), no dice. It will not connect. Still plugging away to find the root issue. – Chris Fellows Mar 30 '15 at 19:26
Finally figured it out. Even though the docs say that Dockerfile will take priority over Dockerrun.aws.conf, something from the Dockerrun was messing things up. I simply took it out and it all worked. I have no idea what was messing things up in there. Here is a [blog article](http://www.oceantara.com/aws-beanstalk-docker-ngnix-debugging-and-troubleshooting/) I put together with some key logging/config files. Hopefully will serve someone out there. – Chris Fellows Mar 31 '15 at 16:43
@ChrisFellows that's awesome! – ericraio Mar 31 '15 at 16:43

score 1 · Answer 2 · answered Aug 16 '19 at 14:36

As mentioned above, the container may have failed. The deployment script is not able to find the IP to which NGINX must forward calls.

So this exception is not the original cause. In order to know what originally happened:

Connect on your EC2 instance with SSH
sudo docker ps -a
Identify which container id is associated to your application (the command is a good way to find it).
sudo docker logs [CONTAINER ID]

I found the answer on this page: https://forums.aws.amazon.com/thread.jspa?messageID=626464

Thank you! In my case, the container was refusing to shut down gracefully, so the deployment command to start a new container was failing due to a lack of memory. A reboot solved the issue (temporarily). — Jonathan Richards, Dec 10 '19 at 19:56

How do you deploy properly to AWS with Elastic beanstalk and Dockerrun.aws.json?

2 Answers2