Docker container builds on OSX but not Amazon Linux

Question

My Docker container builds fine on OSX:

Docker version 17.12.0-ce, build c97c6d6  
docker-compose version 1.18.0, build 8dd22a9

But doesn't build on Amazon Linux:

Docker version 17.12.0-ce, build 3dfb8343b139d6342acfd9975d7f1068b5b1c3d3  
docker-compose version 1.20.1, build 5d8c71b

Full Dockerfile:

# Specify base image
FROM andreptb/oracle-java:8-alpine

# Specify author / maintainer
MAINTAINER Douglas Duhaime <douglas.duhaime@gmail.com>

# Add source to a directory and use that directory
# NB: /app is a reserved directory in tomcat container
ENV APP_PATH="/lts-app"
RUN mkdir "$APP_PATH"
ADD . "$APP_PATH"
WORKDIR "$APP_PATH"

##
# Build BlackLab
##

RUN apk add --update --no-cache \
  wget \
  tar \
  git

# Store the path to the maven home
ENV MAVEN_HOME="/usr/lib/maven"

# Add maven and java to the path
ENV PATH="$MAVEN_HOME/bin:$JAVA_HOME/bin:$PATH"

# Install Maven
RUN MAVEN_VERSION="3.3.9" && \
  cd "/tmp" && \
  wget "http://archive.apache.org/dist/maven/maven-3/$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz" -O - | tar xzf - && \
  mv "/tmp/apache-maven-$MAVEN_VERSION" "$MAVEN_HOME" && \
  ln -s "$MAVEN_HOME/bin/mvn" "/usr/bin/mvn" && \
  rm -rf "/tmp/*"

# Get the BlackLab source
RUN git clone "git://github.com/INL/BlackLab.git"

# Build BlackLab with Maven
RUN cd "BlackLab" && \
  mvn clean install

##
# Build Python + Node dependencies
##

# Install system deps with Alpine Linux package manager
RUN apk add --update --no-cache \
  g++ \
  gcc \
  make \
  openssl-dev \
  python3-dev \
  python \
  py-pip \
  nodejs

# Install Python dependencies
RUN pip install -r "requirements.txt" && \
  npm install --no-optional && \
  npm run build

# Store Mongo service name as mongo host
ENV MONGO_HOST=mongo_service
ENV TOMCAT_HOST=tomcat_service
ENV TOMCAT_WEBAPPS=/tomcat_webapps/

# Make ports available
EXPOSE 7082

# Seed the db
CMD npm run seed && \
  gunicorn -b 0.0.0.0:7082 --access-logfile - --reload server.app:app

Full docker-compose.yml

version: '2'

services:
  tomcat_service:
    image: 'bitnami/tomcat:latest'
    ports:
      - '8080:8080'
    volumes:
      - docker-data-tomcat:/bitnami/tomcat/data/
      - docker-data-blacklab:/lts-app/lts/

  mongo_service:
    image: 'mongo'
    command: mongod
    ports:
      - '27017:27017'

  web:
    # gain access to linked containers
    links:
      - mongo_service
      - tomcat_service
    # explicitly declare service dependencies
    depends_on:
      - mongo_service
      - tomcat_service
    # set environment variables
    environment:
      PYTHONUNBUFFERED: 'true'
    # use the image from the Dockerfile in the cwd
    build: .
    ports:
      - '7082:7082'
    volumes:
      - docker-data-tomcat:/tomcat_webapps
      - docker-data-blacklab:/lts-app/lts/


volumes:
  docker-data-tomcat:
  docker-data-blacklab:

The command I'm running is: docker-compose up --build

The result on Amazon Linux is:

 Running setup.py install for pymongo: started
    Running setup.py install for pymongo: finished with status 'done'
  Running setup.py install for pluggy: started
    Running setup.py install for pluggy: finished with status 'done'
  Running setup.py install for coverage: started
    Running setup.py install for coverage: finished with status 'done'
Successfully installed Faker-0.8.12 Flask-0.12.2 Flask-Cors-3.0.3 Jinja2-2.10 MarkupSafe-1.0 Werkzeug-0.14.1 astroid-1.6.2 attrs-17.4.0 backports.functools-lru-cache-1.5 beautifulsoup4-4.5.1 click-6.7 configparser-3.5.0 coverage-4.5.1 enum34-1.1.6 funcsigs-1.0.2 futures-3.2.0 gunicorn-19.7.1 ipaddress-1.0.19 isort-4.3.4 itsdangerous-0.24 lazy-object-proxy-1.3.1 mccabe-0.6.1 more-itertools-4.1.0 pluggy-0.6.0 py-1.5.3 py4j-0.10.6 pylint-1.8.3 pymongo-3.6.1 pytest-3.5.0 pytest-cov-2.5.1 python-dateutil-2.7.2 singledispatch-3.4.0.3 six-1.11.0 text-unidecode-1.2 wrapt-1.10.11
You are using pip version 8.1.2, however version 9.0.3 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
npm WARN deprecated redux-mock-store@1.5.1: breaking changes in minor version

> base62@1.2.7 postinstall /lts-app/node_modules/base62
> node scripts/install-stats.js || exit 0



ERROR: Service 'web' failed to build: The command '/bin/sh -c pip install -r "requirements.txt" &&   npm install --no-optional &&   npm run build' returned a non-zero code: 1

Does anyone know what might be causing this discrepancy? The error message from Docker doesn't give many clues. I'd be very grateful for any ideas others can offer!

Docker has a logs command you can try running when running your build. You should run that command and post any stacktraces you receive on your web container. That will give you a better idea as to whats happening — ryekayo, Mar 26 '18 at 20:43
thanks @ryekayo! I'm building the container with docker-compose, which seems to have a logs command that can be run after running the up command. Is there a way to fetch/display logs while building a container with docker-compose? — duhaime, Mar 26 '18 at 20:47
I think just capturing the container ID should do it.. When you run docker compose, you should run docker ps -a and see what container is associated w/ your web, then run docker logs and that should do it. Its been a little while since I used docker-compose though but hopefully you can get something.. — ryekayo, Mar 26 '18 at 20:48
Also, just found this: https://stackoverflow.com/questions/37195222/how-to-view-log-output-using-docker-compose-run?utm_medium=organic&utm_source=google_rich_qa&utm_campaign=google_rich_qa This may help — ryekayo, Mar 26 '18 at 20:51
i dont understand why people keep building production images on production machines. I think you are better building your image yourself (or better with CI) and then test that and deploy that.... you might be missing just some dependencies that exist on your PC and not on amazon, building the image requires lot of system resources and you just run it on production and might affect your APP performance — Mazel Tov, Mar 27 '18 at 08:09

score 0 · Accepted Answer · answered Mar 29 '18 at 19:04

To solve this problem, I followed @MazelTov's advice and built the containers on my local OSX development machine, then published the images to Docker Cloud, then pulled those images down onto and ran the images from my production server (AWS EC2).

Install Dependencies

I'll try and outline the steps I followed below in case they help others. Please note these steps require you to have docker and docker-compose installed on your development and production machines. I used the gui installer to install Docker for Mac.

Build Images

After writing a Dockerfile and docker-compose.yml file, you can build your images with docker-compose up --build.

Upload Images to Docker Cloud

Once the images are built, you can upload them to Docker Cloud with the following steps. First, create an account on Docker Cloud.

Then store your Docker Cloud username in an environment variable (so your ~/.bash_profile should contain export DOCKER_ID_USER='yaledhlab' (use your username though).

Next login to your account from your developer machine:

docker login

Once you're logged in, list your docker images:

docker ps

This will display something like:

CONTAINER ID        IMAGE                          COMMAND                  CREATED             STATUS              PORTS                      NAMES
89478c386661        yaledhlab/let-them-speak-web   "/bin/sh -c 'npm run…"   About an hour ago   Up About an hour    0.0.0.0:7082->7082/tcp     letthemspeak_web_1
5e9c75d29051        training/webapp:latest         "python app.py"          4 hours ago         Up 4 hours          0.0.0.0:5000->5000/tcp     heuristic_mirzakhani
890f7f1dc777        bitnami/tomcat:latest          "/app-entrypoint.sh …"   4 hours ago         Up About an hour    0.0.0.0:8080->8080/tcp     letthemspeak_tomcat_service_1
09d74e36584d        mongo                          "docker-entrypoint.s…"   4 hours ago         Up About an hour    0.0.0.0:27017->27017/tcp   letthemspeak_mongo_service_1

For each of the images you want to publish to Docker Cloud, run:

docker tag image_name $DOCKER_ID_USER/my-uploaded-image-name
docker push $DOCKER_ID_USER/my-uploaded-image-name

For example, to upload mywebapp_web to your user's account on Docker cloud, you can run:

docker tag mywebapp_web $DOCKER_ID_USER/web
docker push $DOCKER_ID_USER/web

You can then run open https://cloud.docker.com/swarm/$DOCKER_ID_USER/repository/list to see your uploaded images.

Deploy Images

Finally, you can deploy your images on EC2 with the following steps. First, install Docker and Docker-Compose on the Amazon-flavored EC2 instance:

# install docker
sudo yum install docker -y

# start docker
sudo service docker start

# allow ec2-user to run docker
sudo usermod -a -G docker ec2-user

# get the docker-compose binaries
sudo curl -L https://github.com/docker/compose/releases/download/1.20.1/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose

# change the permissions on the source
sudo chmod +x /usr/local/bin/docker-compose

Log out, then log back in to update your user's groups. Then start a screen and run the server: screen. Once the screen starts, you should be able to add a new docker-compose config file that specifies the path to your deployed images. For example, I needed to fetch the let-them-speak-web container housed within yaledhlab's Docker Cloud account, so I changed the docker-compose.yml file above to the file below, which I named production.yml:

version: '2'

services:
  tomcat_service:
    image: 'bitnami/tomcat:latest'
    ports:
      - '8080:8080'
    volumes:
      - docker-data-tomcat:/bitnami/tomcat/data/
      - docker-data-blacklab:/lts-app/lts/

  mongo_service:
    image: 'mongo'
    command: mongod
    ports:
      - '27017:27017'

  web:
    image: 'yaledhlab/let-them-speak-web'
    # gain access to linked containers
    links:
      - mongo_service
      - tomcat_service
    # explicitly declare service dependencies
    depends_on:
      - mongo_service
      - tomcat_service
    # set environment variables
    environment:
      PYTHONUNBUFFERED: 'true'
    ports:
      - '7082:7082'
    volumes:
      - docker-data-tomcat:/tomcat_webapps
      - docker-data-blacklab:/lts-app/lts/

volumes:
  docker-data-tomcat:
  docker-data-blacklab:

Then the production compose file can be run with: docker-compose -f production.yml up. Finally, ssh in with another terminal, and detach the screen with screen -D.

Docker container builds on OSX but not Amazon Linux

1 Answers1

Install Dependencies

Build Images

Upload Images to Docker Cloud

Deploy Images