Deploying simple docker app with docker-compose on Elastic Beanstalk

Question

I have a simple docker app that is able to run for me locally via docker-compose up, and when I send the .yml file to my friend, they are also able to get it up and running on their local machine. However, when I try to deploy it on Elastic Beanstalk, I get errors (specifically, something related to error:open /var/pids/eb-docker-compose-log.pid: no such file or directory, as I'll show below). I've tried to upload multiple times to Elastic Beanstalk, with the same errors. This is a custom app, but they are the same errors I got when I was trying to follow the instructions on https://docker-curriculum.com/#docker-on-aws. Here is the docker-compose.yml for my current app:

version: "3"
services:
  server:
    image: mfatigati/shop-server
    container_name: shop-server
    ports:
      - "4000:4000"
  client:
    image: mfatigati/shop-client
    depends_on:
      - server
    ports:
      - "3000:3000"

mfatigati/shop-server and mfatigati/shop-client are both Node.JS apps, i.e., FROM node:16 in their Dockerfile.

To deploy this on AWS, I go to my EB console, and then:

1. Click "Create Application" to take me to the create app screen
2. Choose "Docker" as the platform
3. Choose "Upload local code", and upload the above-mentioned .yml file.
4. Click "Create Application"

Based on the notes here, I think this should be all I need to do (maybe I'm wrong about that?), but I get errors every time that point me to the eb.engine.log file. I've pasted what seems to be the relevant section below, as it is the only section that mentions errors, and it also reflects what appears in the AWS GUI console. The main problem seems reflected by the bit about error:open /var/pids/eb-docker-compose-log.pid: no such file or directory:

2022/02/14 14:17:23.619888 [ERROR] update processes [cfn-hup eb-docker-events healthd eb-docker-compose-events eb-docker-compose-log docker] pid symlinks failed with error Read pid source file /var/pids/eb-docker-compose-log.pid failed with error:open /var/pids/eb-docker-compose-log.pid: no such file or directory
2022/02/14 14:17:23.619901 [ERROR] An error occurred during execution of command [app-deploy] - [Track pids in healthd]. Stop running the command. Error: update processes [cfn-hup eb-docker-events healthd eb-docker-compose-events eb-docker-compose-log docker] pid symlinks failed with error Read pid source file /var/pids/eb-docker-compose-log.pid failed with error:open /var/pids/eb-docker-compose-log.pid: no such file or directory 

2022/02/14 14:17:23.619905 [INFO] Executing cleanup logic
2022/02/14 14:17:23.620005 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1644848243,"severity":"ERROR"}]}]}

Any insight would be greatly appreciated! I'm pasting some screenshots below, in case that helps.

GUI corresponding to step 2; GUI corresponding to step 3; GUI errors

Answer 1

I had the same error:open /var/pids/eb-docker-compose-log.pid: no such file or directory error happening for my Docker Compose app when trying to deploy it to my Elastic Beanstalk environment; I'm not sure if my solution will be the same solution you need, but I hope it points you in the right direction (and helps future devs facing a similar problem).

---

What caused the error for me:

This ...eb-docker-compose-log.pid: no such file... error was a false error that was triggered by a separate issue; my separate error was actually a problem with my application code not finding the environment variables set in my Elastic Beanstalk environment. See below for how I found the problem, and what I did to fix it.

---

How I found my real problem:

I downloaded the Full Logs:

1. go to your EB environment
2. click Logs on the left nav
3. click the Request Logs dropdown button (at the top right)
4. click Full Logs
5. click the Download link once the full logs are ready

Inside of the logs, I found the real problem in the eb-docker/containers/eb-current-app/eb-stdouterr.log file, the issue being that my application code wasn't able to find the environment variables that were setup in my Elastic Beanstalk Software configuration.

In case you're curious what my error said:

panic: required key ONE_OF_MY_ENV_KEYS missing value

(I also had a couple other errors in this log that I fixed, but fixing the error shown above is what ended up solving the ...eb-docker-compose-log.pid: no such file... error).

---

How I fixed this error:

I turns out that if you use docker-compose.yml, while setting up your environment variables in your Elastic Beanstalk Software configuration, you have to make sure you use the .env file that Elastic Beanstalk creates for you; otherwise (from my own testing), EB only see's/uses the environment variable keys/values you specify in your own .env file or environment: list you can specify in docker-compose.yml.

NOTE: see the Elastic Beanstalk "Environment properties and Environment Variables" and "Referencing environment variables in containers" sections in the docs here, in particular this bit:

"Elastic Beanstalk generates a Docker Compose environment file called .env in the root directory of your application project. This file stores the environment variables you configured for Elastic Beanstalk.

Note If you include a .env file in your application bundle, Elastic Beanstalk will not generate an .env file."

I solved my problem by updating my docker-compose.yml file to point to the supposed .env file that EB would create for me (by adding env_file: .env to my services that needed it), i.e.:

version: "3"
services:
  my_service1:
    # ...
    env_file: .env
  my_service2:
    # ...
    env_file: .env