How to pass two files from different directory to docker run?

Question

I have a docker container that looks like this

FROM python:3.8

ADD lodc.py .

RUN pip install requests python-dotenv

CMD [ "python", "./lodc.py", "file1.json", "file2.json" ]

it needs to take an env file and then two different files as the arguments that is needed for the script lodc.py to run. I have tried mounting them like described here Passing file as argument to Docker container but I cannot get it to work. It is important to keep the two files isolated because those files will be changing frequently so it doesn't make sense to put them into the container. Here is what I've been running

docker run --env-file /Users/Documents/github/datasets/tmp/.env -v /Users/Documents/datasets/files:/Users/Documents/github/datasets datasets datasets/file1.json datasets/file2.json

Basically I would like to just build and run the docker container and be able to munipulate the two argument files in another directory whenever I want without issue.

The env file is being passed correctly and it is failing because it cannot find the file.json directory. I am new to docker and any help would be greatly appreciated. Thanks.

I think it just a path problem. try using the full paths for the arguments like `/Users/Documents/github/datasets/file1.json` — pbacterio, Aug 11 '21 at 15:08
@pbacterio when doing that it gives me no such file or directory — Steve, Aug 11 '21 at 15:10

tomarv2 · Answer 1 · 2021-08-11T16:01:49.370

1

If the one you are trying does not work. You can also try below as an option(keep in mind that you will have to build docker image every time input files change):

I created the below directory structure based on your comments:

FROM python:3.8

ADD lodc.py .

COPY dir1/file1.json .
COPY dir2/file2.json .

CMD [ "python", "./lodc.py", "file1.json", "file2.json"]

Please let me know if you need more help.

edited Aug 11 '21 at 16:01

answered Aug 11 '21 at 15:23

tomarv2

753
3
14

He doesn't need to copy them in docker build if he has them as a volume. – czende Aug 11 '21 at 15:30

score 1 · Accepted Answer · answered Aug 11 '21 at 15:28

I think you just got the run command wrong. Try this one:

docker run \
--env-file /Users/Documents/github/datasets/tmp/.env \
-v /Users/Documents/datasets/files/file1.json:/file1.json \ 
-v /Users/Documents/github/datasets/files/file2.json:/file2.json \
<your-built-docker-image-name>

I'm not sure about your paths, but you need to run it with two different volumes (-v argument).

score 0 · Answer 3 · edited Aug 11 '21 at 15:52

You just have to make clear where you are running your python (i.e, where are "you" inside the container).

In other words, you have to define (somehow) your WORKDIR. By default, the workdir is at / (the container's root directory).

Let's suppose, for simplicity, that you define your workdir to be inside your "datasets/files" directory. (Also, I'll use a shorter path inside the container...because we can ;)

So, I'd suggest the following for your Dockerfile and docker-run:

FROM python:3.8

ADD lodc.py .

RUN pip install requests python-dotenv

CMD [ "python", "/lodc.py", "file1.json", "file2.json" ]

build:

$ docker build -t datasets

run:

$ docker run --env-file /Users/Documents/github/datasets/tmp/.env \
    -v /Users/Documents/datasets/files:/mnt/datasets \
    -w /mnt/datasets
    datasets

Or, you could free your Dockerfile a little bit, and push it to your run command:

FROM python:3.8

ADD lodc.py .

RUN pip install requests python-dotenv

ENTRYPOINT [ "python", "/lodc.py" ]

build:

$ docker build -t datasets

run:

$ docker run --env-file /Users/Documents/github/datasets/tmp/.env \
    -v /Users/Documents/datasets/files:/mnt/datasets \
    datasets /mnt/datasets/file1.json /mnt/datasets/file2.json

I didn't test it, but it looks (syntactically) ok.

score 0 · Answer 4 · answered Aug 11 '21 at 15:49

To give a complete answer you can do it three ways:

In the Docker file you can use the Copy command - this is nice, but it kind of hides what volumes are being used if you use an orchestration tool like Kubernetes or Docker-Compose down the line

FROM python:3.8

ADD lodc.py .

COPY dir1/file1.json .
COPY dir2/file2.json .

CMD [ "python", "./lodc.py", "file1.json", "file2.json"]

You can reference a volume in command line using the -v argument; this is less preferable as it doesn't codify the volumes and it is dependent on the directory the command is executed in

docker run \
    --env-file /Users/Documents/github/datasets/tmp/.env \
    -v /Users/Documents/datasets/files/file1.json:/file1.json \ 
    -v /Users/Documents/github/datasets/files/file2.json:/file2.json \
    <your-built-docker-image-name>

You can reference a volume in a docker-compose file; this option is preferred because the volume referenced is in code, it is explicit the volume is included, and it isn't dependent on where the command is executed

version: '3.9'
services:

  base:
    build:
      context: .
      dockerfile: <relative-path-to-dockerfile-from-docker-compose>
    image: <desired-image-name>
    env_file:
      - /Users/Documents/github/datasets/tmp/.env
    volumes:
      - /Users/Documents/datasets/files/file1.json:/file1.json
      - /Users/Documents/github/datasets/files/file2.json:/file2.json

The last option will reduce headaches in the long-term as you scale because is the most explicit and stable; Good Luck!

How to pass two files from different directory to docker run?

4 Answers4