8

My project directory structure:

myapp/
    src/
    Dockerfile
    docker-compose.yml
    docker-deploy.sh
    wait-for-it.sh
    .env

Where wait-for-it.sh is a copy of the famous wait-for-it script.

My Dockerfile:

FROM node:16

WORKDIR /usr/src/app

COPY package*.json ./
COPY wait-for-it.sh ./
COPY docker-deploy.sh ./

RUN chmod +x docker-deploy.sh

RUN npm install --legacy-peer-deps

COPY . .

RUN npm run build

ENTRYPOINT ["docker-deploy.sh"]

And docker-deploy.sh is:

#!/bin/bash

# make wait-for-it executable
chmod +x wait-for-it.sh

# call wait-for-it with passed in args and then start node if it succeeds
bash wait-for-it.sh -h $1 -p $2 -t 300 -s -- node start

And my docker-compose.yml:

version: '3.7'

services:
  my-service:
    build: .
  postgres:
    container_name: postgres
    image: postgres:14.3
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USER}
      POSTGRES_DB: my-service-db
      PG_DATA: /var/lib/postgresql2/data
    ports:
      - ${DB_PORT}:${DB_PORT}
    volumes:
      - pgdata:/var/lib/postgresql2/data
volumes:
  pgdata:

And where my .env looks like:

DB_PASSWORD=1234
DB_USER=root
DB_PORT=5432

When I run the following command-line from the project root:

docker-compose --env-file .env up --build

I get:

Creating myapp_my-service_1 ... error

Creating postgres                    ... 
Creating postgres                    ... done

ERROR: for my-service  Cannot start service my-service: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "docker-deploy.sh": executable file not found in $PATH: unknown

ERROR: Encountered errors while bringing up the project.

What is going on? Is the error coming from the wait-for-it.sh script itself, from a poorly configured CMD directive in the Dockerfile, or from the actual Node/JS app running as my-service?

Update

Latest errors after applying @ErikMD's suggested changes:

Creating postgres ... done
Creating myapp_my-service_1 ... error

ERROR: for myapp_my-service_1  Cannot start service my-service: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "./docker-deploy.sh": permission denied: unknown

ERROR: for my-service  Cannot start service my-service: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "./docker-deploy.sh": permission denied: unknown
ERROR: Encountered errors while bringing up the project.

So it is spinning up the DB (postgres) no problem but is still for some reason getting permissions-related issues with the docker-deploy.sh script.

hotmeatballsoup
  • 385
  • 6
  • 58
  • 136
  • This actually seems like your container is trying to run something like `node ./wait-for-it.sh` Can you try using `ENTRYPOINT` instead of `CMD`? – derpirscher Aug 19 '22 at 09:26
  • Also did you check if your `wait-for-it.sh` is actually an executable? If not, `CMD ["./wait-for-it.sh", ...]` might not be interpreted correctly as "exec form" of [CMD](https://docs.docker.com/engine/reference/builder/#cmd) but as "param from" thus, all the parameters passed on to the default `ENTRYPOINT` of the container which is the `node` executable (and which would fit the observed behaviour) – derpirscher Aug 19 '22 at 09:32
  • Thanks I've tried making it executable as well as introducing a `docker-deploy.sh` script and invoking that from an `ENTRYPOINT`, which seems to be a common-ish practice, but still problems! – hotmeatballsoup Aug 23 '22 at 18:10
  • @hotmeatballsoup after your edit, your `ENTRYPOINT` looks weird; please try `ENTRYPOINT ["./docker-deploy.sh"]`. – ErikMD Aug 23 '22 at 22:09
  • Also don't forget the `exec` builtin at the end of the entrypoint (useful to avoid [typical issues](https://stackoverflow.com/questions/50534605/speed-up-docker-compose-shutdown/)) → `exec ./wait-for-it.sh -h $1 -p $2 -t 300 -s -- node start`. – ErikMD Aug 23 '22 at 22:14
  • 1
    found [this](https://serverfault.com/questions/772227/chmod-not-working-correctly-in-docker) and [this](https://github.com/moby/moby/issues/29922) that could explain the issue. You should check if the workaround described works in your case. – Lety Aug 28 '22 at 10:56

2 Answers2

5

As pointed out in @derpirscher's comment and mine, one of the issues was the permission of your script(s) and the way they should be called as the ENTRYPOINT (not CMD).

Consider this alternative code for your Dockerfile :

FROM node:16

WORKDIR /usr/src/app

COPY package*.json ./
COPY wait-for-it.sh ./
COPY docker-deploy.sh ./

# Use a single RUN command to avoid creating multiple RUN layers
RUN chmod +x wait-for-it.sh \
  && chmod +x docker-deploy.sh \
  && npm install --legacy-peer-deps

COPY . .

RUN npm run build

ENTRYPOINT ["./docker-deploy.sh"]

docker-deploy.sh script :

#!/bin/sh

# call wait-for-it with args and then start node if it succeeds
exec ./wait-for-it.sh -h "${DB_HOST}" -p "${DB_PORT}" -t 300 -s -- node start

See this other SO question for more context on the need for the exec builtin in a Docker shell entrypoint.

Also, note that the fact this exec ... command line is written inside a shell script (not directly in an ENTRYPOINT / CMD exec form) is a key ingredient for using the parameter expansion.
In other words: in the revision 2 of your question, the "${DB_HOST}:${DB_PORT}" argument was understood literally because no shell interpolation occurs in an ENTRYPOINT / CMD exec form.

Regarding the docker-compose.yml :

# version: '3.7'
# In the Docker Compose specification, "version:" is now deprecated.

services:
  my-service:
    build: .
    # Add "image:" for readability
    image: some-optional-fresh-tag-name
    # Pass environment values to the entrypoint
    environment:
      DB_HOST: postgres
      DB_PORT: ${DB_PORT}
      # etc.
    # Add network spec to make it explicit what services can communicate together
    networks:
      - postgres-network
    # Add "depends_on:" to improve "docker-run scheduling":
    depends_on:
      - postgres

  postgres:
    # container_name: postgres # unneeded
    image: postgres:14.3
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USER}
      POSTGRES_DB: my-service-db
      PG_DATA: /var/lib/postgresql2/data
    volumes:
      - pgdata:/var/lib/postgresql2/data
    networks:
      - postgres-network
    # ports:
    #   - ${DB_PORT}:${DB_PORT}
    # Rather remove this line in prod, which is a typical weakness, see (§)

networks:
  postgres-network:
    driver: bridge

volumes:
  pgdata:
    # let's be more explicit
    driver: local

Note that in this Docker setting, the wait-for-it host should be postgres (the Docker service name of your database), not 0.0.0.0 nor localhost. Because the wait-for-it script acts as a client that tries to connect to the specified web service in the ambient docker-compose network.

For a bit more details on the difference between 0.0.0.0 (a server-side, catch-all special IP) and localhost in a Docker context, see e.g. this other SO answer of mine.

(§): last but not least, the ports: [ "${DB_PORT}:${DB_PORT}" ] lines should rather be removed because they are not necessary for the Compose services to communicate (the services just need to belong to a common Compose network and use the other Compose services' hostname), while exposing one such port directly on the host increases the attack surface.

Last but not least:

To follow-up this comment of mine, suggesting to run ls -l docker-deploy.sh; file docker-deploy.sh in your myapp/ directory as a debugging step (BTW: feel free to do this later on then comment for the record):

Assuming there might be an unexpected bug in Docker similar to this one as pointed by @Lety:

I'd suggest to just replacing (in the Dockerfile) the line

RUN chmod +x wait-for-it.sh \
  && chmod +x docker-deploy.sh \
  && npm install --legacy-peer-deps

with

RUN npm install --legacy-peer-deps

and running directly in a terminal on the host machine:

cd myapp/
chmod -v 755 docker-deploy.sh
chmod -v 755 wait-for-it.sh

docker-compose --env-file .env up --build

If this does not work, here is another useful information you may want to provide: what is your OS, and what is your Docker package name? (e.g. docker-ce or podman…)

ErikMD
  • 13,377
  • 3
  • 35
  • 71
  • It's not sane to recommend the executable `/usr/bin/env` nor the bash shell for an in container environment. The common expectation is `/bin/sh` to exist (for linux containers, and who wants to use anything else, right?). Parameter expansion is available within the Dockerfile as well, both on the level of the file with build parameters (which docker-compose does play well with) as well as the container environment during the build. Just saying. – hakre Aug 24 '22 at 00:24
  • Thanks @ErikMD (+1) for an amazing answer. I applied all of your changes and ran `docker-compose --env-file .env up --build` and got new errors, please see my update above! Looks like `docker-deploy.sh` is still getting permission-related errors. Maybe something like a `chmod 400` as well? Thanks again so much! – hotmeatballsoup Aug 24 '22 at 10:26
  • Or maybe a `sudo exec ...` to invoke `wait-for-it.sh`? I wonder what OS user is running the `exec` currently? – hotmeatballsoup Aug 24 '22 at 10:32
  • @hakre I believe what you suggest is both correct (the fact that `/bin/sh` always is available) and subjective (the fact that we should always avoid `bash`), indeed given the OP has full control on the base image, it is easy to check that `bash` is available in `node:16`. However I agree that using either `/bin/bash` or `/usr/bin/env bash` (often [recommended](https://mywiki.wooledge.org/BashProgramming#Shebang) to maximize portability) is equivalent, and that the former is much simpler… Will edit my answer accordingly. – ErikMD Aug 24 '22 at 11:37
  • @hotmeatballsoup I cannot reproduce your error, except if I comment-out the `RUN chmod +x docker-deploy.sh` command in my minimal-reproducible-example Dockerfile (→ I then get « docker: Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "./docker-deploy.sh": permission denied: unknown. »). So, can you double-check that you indeed run this command in your Dockerfile, and that you test the proper image? (BTW, what is your OS and Docker package name?) – ErikMD Aug 24 '22 at 11:42
  • @ErikMD: If its your own Dockerfile (build), you can use any of those, but as an answer to the question it is just not portable. That's all. /bin/bash vs. /usr/bin/env bash consult the code-style requirements. I would be a bit less enthusiastic and call the later a recommendation. I don't use it for bash (just `/bin/bash` then), but I use it for php (that is `/usr/bin/env php` then). But as its subjective, for Q&A material I think portability is a gain. – hakre Aug 24 '22 at 11:46
  • @hotmeatballsoup *I wonder what OS user is running the exec currently?* inside the container, the OS is `Debian GNU/Linux 10 (buster)` (to get this, I just added `RUN cat /etc/os-release`), and [`exec`](https://mywiki.wooledge.org/BashSheet#Execution) is a Bash builtin which means "pass the control to the specified program" (i.e., replace the current shell with this program, instead of spinning a new process). – ErikMD Aug 24 '22 at 11:48
  • @hotmeatballsoup: Regarding the user: The default user is configured in the container. Often `root` (users are a shared resource between containers and hosts unless user namespacing is involved). You can as well execute `whoami` in your build or the `id` utility to learn more. The build will show this to you then. Use the `RUN` for your benefit. – hakre Aug 24 '22 at 11:50
  • @hakre OK, I agree that for this specific answer, it's better to recommend a POSIX shebang as we don't need bash features here. Will edit this anew in my answer :) (But I don't follow completely you when you say that bash is not portable *and thereby shouldn't be used in a Docker context*, e.g. when we really need bash features in an Alpine container, we can just do `apk add --no-cache bash` and here we go ;) – ErikMD Aug 24 '22 at 11:55
  • @hotmeatballsoup to debug things more easily and check permissions have been correctly applied, could you add just before the `ENTRYPOINT` a command such as `RUN ls -hal *.sh` ? – ErikMD Aug 24 '22 at 11:58
  • @ErikMD thanks for all the help here. I have double checked that I implemented your suggestions correctly. When I run docker-compose I see `=> CACHED [6/9] RUN chmod +x wait-for-it.sh && chmod +x docker-deploy.sh && npm install --legacy-peer-deps` and then a few lines later `=> [9/9] RUN ls -hal *.sh`. However it failing with the same permission denied error prior to having a chance to display anything that the `ls -hal *.sh` would print. If this doesn't spur any ideas for you, then tonight I will push a fully-reproducible project to GitHub and I'll link it here. Thanks again! – hotmeatballsoup Aug 24 '22 at 12:31
  • @hotmeatballsoup OK but normally, the output of `ls -hal *.sh` should be *part of the build log* (not at `docker run`time). So if you see nothing, that's already a hint(?) – ErikMD Aug 24 '22 at 12:33
  • BTW, do you compile with `docker-compose build` or `docker-compose up --build` ? (because `docker-compose up` wouldn't be enough) – ErikMD Aug 24 '22 at 12:35
  • The full command I'm using is `docker-compose --env-file .env up --build` but let me know if that needs to be changed/tweaked! – hotmeatballsoup Aug 24 '22 at 12:36
  • OK! That's fine, this command looks good. – ErikMD Aug 24 '22 at 12:37
  • BTW could you try running these two commands in your host (outside of Docker)? `ls -l docker-deploy.sh` and `file docker-deploy.sh` – ErikMD Aug 24 '22 at 12:44
  • @hotmeatballsoup did you get a chance to test the final version of my answer? – ErikMD Aug 31 '22 at 16:24
1

It's a bit like in the shell on your computer: You enter the command to execute (just a basename of the command) and the shell tells you if it can't find it - except it won't tell you its not withing $PATH as it could be (compare hash utility).

Now Docker ain't a shell and therefore the message is a bit more verbose (and that docker-compose is running docker is also adding in front of it):

ERROR: for my-service Cannot start service my-service: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "docker-deploy.sh": executable file not found in $PATH: unknown

So the part of Docker is:

OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "docker-deploy.sh": executable file not found in $PATH: unknown

And that is effectively due to the commandment in the Dockerfile:

ENTRYPOINT ["docker-deploy.sh"]

Whatever the (absolute) path of docker-deploy.sh is within the container, the basename of it (docker-deploy.sh, again) could not be found within the containers environment PATH parameter (compare PATH (in), Pathname Resolution, etc.).

Use the basename of an executable that is actually in PATH within the container or an absolute (or relative to the containers PWD environment parameter a.k.a. working directory) so that it actually can be executed (by Docker).

hakre
  • 193,403
  • 52
  • 435
  • 836