2

I have a scrapy project run continously by cron hosted inside a docker image.

When I run and deploy this locally everything works fine. If I try to deploy the same to AWS I get the following error inside the logs:

No EXPOSE directive found in Dockerfile, abort deployment (ElasticBeanstalk::ExternalInvocationError)

The console shows that my container was build correctly but I can not use it without an EXPOSED port.

INFO: Successfully pulled python:2.7
WARN: Failed to build Docker image aws_beanstalk/staging-app, retrying...
INFO: Successfully built aws_beanstalk/staging-app
ERROR: No EXPOSE directive found in Dockerfile, abort deployment
ERROR: [Instance: i-6eebaeaf] Command failed on instance. Return code: 1 Output: No EXPOSE directive found in Dockerfile, abort deployment.
Hook /opt/elasticbeanstalk/hooks/appdeploy/enact/00run.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
INFO: Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].

But why is it not possible?

My Dockerfile looks like the following:

FROM python:2.7
MAINTAINER XDF

ENV DIRECTORY /opt/the-flat

# System
##########

RUN apt-get update -y && apt-get upgrade -y && apt-get install -y ntp vim apt-utils
WORKDIR $DIRECTORY

# GIT
##########
# http://stackoverflow.com/questions/23391839/clone-private-git-repo-with-dockerfile

RUN apt-get install -y git
RUN mkdir /root/.ssh/
ADD deploy/git-deply-key /root/.ssh/id_rsa
RUN chmod 0600 /root/.ssh/id_rsa
RUN touch /root/.ssh/known_hosts
RUN ssh-keyscan -t rsa bitbucket.org >> /root/.ssh/known_hosts
RUN ssh -T -o 'ConnectionAttempts=1' git@bitbucket.org
RUN git clone --verbose git@bitbucket.org:XDF/the-flat.git .

# Install
##########

RUN pip install scrapy
RUN pip install MySQL-python

# not working
# apt-get install -y wkhtmltopdf && pip install pdfkit
# else
# https://pypi.python.org/pypi/pdfkit

RUN DEBIAN_FRONTEND=noninteractive apt-get install -y openssl build-essential xorg libssl-dev
RUN wget http://wkhtmltopdf.googlecode.com/files/wkhtmltopdf-0.10.0_rc2-static-amd64.tar.bz2
RUN tar xvjf wkhtmltopdf-0.10.0_rc2-static-amd64.tar.bz2
RUN chown root:root wkhtmltopdf-amd64
RUN mv wkhtmltopdf-amd64 /usr/bin/wkhtmltopdf
RUN pip install pdfkit

# Cron
##########
# http://www.ekito.fr/people/run-a-cron-job-with-docker/
# http://www.corntab.com/pages/crontab-gui

RUN apt-get install -y cron
RUN crontab "${DIRECTORY}/deploy/crontab"

CMD ["cron", "-f"]
lony
  • 6,733
  • 11
  • 60
  • 92

2 Answers2

6

It's by design. You need to have an EXPOSE port directive in your Dockerfile to tell beanstalk what port your app will be listening on. Do you have a usecase where you cannot or do not want to have EXPOSE in your Dockerfile?

Rohit Banga
  • 18,458
  • 31
  • 113
  • 191
  • Sadly yes, my scrapy crawler just fetches data and stores it into a database. So no exposure needed. For now I just EXPOSED a random port and it worked .. but why is it needed? – lony Aug 30 '15 at 10:12
  • 1
    Typically this is for applications structured as Web applications. Worker tier is also architected as a Web app even though the port is only open on localhost for worker tier. – Rohit Banga Aug 30 '15 at 10:15
  • @RohitBanga We have been running workers in Elastic Beanstalk for a couple years now that do not expose ports -- they connect to a queue services and process messages. This requirement to EXPOSE a port is silly for a WORKER tier container. As a workaround we'll have to EXPOSE some port even though it won't be used. – kingdango Mar 28 '18 at 15:51
  • Its only exposed on localhost because the worker tier daemon is running outside your container. – Rohit Banga Mar 28 '18 at 15:53
1

ElasticBeanstalk is designed for web applications, hence the EXPOSE requirement. The use case you demonstrated is that of a jobs (workers) server, which Elastic Beanstalk doesn't handle well.
For your case, either expose a dummy port number or launch an EC2 instance yourself to bypass the EB overload.

Tal
  • 7,827
  • 6
  • 38
  • 61