Dockerfile COPY instruction failing?

Question

All, i'm trying to persistently copy files from my host to an image so those files are available with every container launched based on that image. Running on debian wheezy 64bit as virtualbox guest.

the Dockerfile is fairly simple (installing octave image):

FROM debian:jessie 
MAINTAINER GG_Python <[redacted]@gmail.com>
RUN apt-get update 
RUN apt-get update
RUN apt-get install -y octave octave-image octave-missing-functions octave-nan octave-statistics

RUN mkdir /octave
RUN mkdir /octave/libs
RUN mkdir /octave/libs/jsonlab
COPY ~/octave/jsonlab/loadjson.m /octave/libs/jsonlab/.

I'm getting the following trace after issuing a build command: docker build -t octave .

Sending build context to Docker daemon 423.9 kB
Sending build context to Docker daemon 
Step 0 : FROM debian:jessie
 ---> 58052b122b60
Step 1 : MAINTAINER GG_Python <[..]@gmail.com>
 ---> Using cache
 ---> 90d2dd2f7ee8
Step 2 : RUN apt-get update
 ---> Using cache
 ---> 4c72c25cd829
Step 3 : RUN apt-get update
 ---> Using cache
 ---> b52f0bcb9f86
Step 4 : RUN apt-get install -y octave octave-image octave-missing-functions octave-nan octave-statistics
 ---> Using cache
 ---> f0637ab96d5e
Step 5 : RUN mkdir /octave
 ---> Using cache
 ---> a2d278b2819b
Step 6 : RUN mkdir /octave/libs
 ---> Using cache
 ---> 65efbbe01c99
Step 7 : RUN mkdir /octave/libs/jsonlab
 ---> Using cache
 ---> e41b80901266
Step 8 : COPY ~/octave/jsonlab/loadjson.m /octave/libs/jsonlab/.
INFO[0000] ~/octave/jsonlab/loadjson.m: no such file or directory

Docker absolutely refuses to copy this file from the host into the image. Needless to say a the file loadjson.m is there (cat displays), all my attempts to change the path (relative, absolute, etc.) failed. Any advice why this simple task is problematic?

With the mkdir you can use the -p option to create them all in one go: mkdir -p /octave/libs/jsonlab — dalore, Dec 21 '17 at 11:32

kojiro · Accepted Answer · 2018-10-12T12:45:14.270

28

At the time I originally wrote this, Docker didn’t expand ~ or $HOME. Now it does some expansions inside the build context, but even so they are probably not what you want—they aren’t your home directory outside the context. You need to reference the file explicitly, or package it relative to the Dockerfile itself.

edited Oct 12 '18 at 12:45

answered Jan 21 '15 at 01:00

kojiro

74,557
19
143
201

1

WORKS! I've copied the file into the same directory as the Dockerfile and was careful with the destination folder (must have backslash but "/." as a destination file fails). It seems like docker is very picky with this operation.. – GG_Python Jan 21 '15 at 01:08
1

Weirdly, using an explicit path to the source file did not work for me - even if it was in the same location as the Dockerfile. I removed the path to the source file and it worked. – ben Jul 28 '16 at 11:12
1

I keep stumbling over this, Docker, why do you make our lives artificially difficult? The file is there, the path is correct, why doesn't this work? Seriously, there are tons of other real problems that need solving, why implement something so horribly error prone? – Markus Bawidamann Mar 22 '21 at 05:36
Always make sure you add the WORKDIR before the copy ```WORKDIR /home/user``` ```COPY ./ .``` – user3613987 Jul 13 '21 at 20:09
Works! I copied to /root/dir instead of ~/dir (running as user root btw) – Marcelo Vani Nov 03 '22 at 17:43
Just remove file path from .dockerignore if you mentioned – Harikrushna Patel Dec 04 '22 at 16:39

score 25 · Answer 2 · answered Jan 21 '15 at 01:05

25

Docker can only copy files from the context, the folder you are minus any file listed in the dockerignore file.

When you run 'docker build' docker tars the context and it sends it to the docker daemon you are connected to. It only lets you copy files inside of the context because the daemon might be a remote machine.

answered Jan 21 '15 at 01:05

Javier Castellanos

9,346
2
15
19

13

This was my case, I was working on an old project, I realized that I had an "*" rule in a `.dockerignore`, thanks a lot – gsalgadotoledo May 03 '17 at 14:57
docker ignore just bit me too, thanks for the info. – steven87vt Aug 18 '20 at 18:56
Literally searched for 3 hours straight now and i thought i was going insane why it would refuse to copy anything. Thanks! – Nestroy May 29 '21 at 13:59

score 9 · Answer 3 · answered May 06 '17 at 09:17

I couldn't get COPY to work until I understood the context (I was trying to copy a file from outside of the context)

The docker build command builds an image from a Dockerfile and a context. The build’s context is the files at a specified location PATH. The PATH is a directory on your local filesystem.

A context is processed recursively. So, a PATH includes any subdirectories.

The build is run by the Docker daemon, not by the CLI. The first thing a build process does is send the entire context (recursively) to the daemon. In most cases, it’s best to start with an empty directory as context and keep your Dockerfile in that directory. Add only the files needed for building the Dockerfile.

Warning: Do not use your root directory, /, as the PATH as it causes the build to transfer the entire contents of your hard drive to the Docker daemon.

Reference: https://docs.docker.com/engine/reference/builder/#usage

I don't get what context means in this. No, it does not send the whole / directory, why would it do that? When you build an image, it is not automatically copying all files, it copies the ones you explicitly copy. I'm currently stuck with not being able to copy ONE file, much less any more. It always throws the error message, refusing to execute the copy command in the docker file, extremely frustrating. — Markus Bawidamann, Mar 22 '21 at 05:46

Ash Singh · Answer 4 · 2020-10-19T02:37:55.053

3

I had similar issue. I solved it by checking two things:

Inside your, docker-compose.yaml check context of the service, docker will not copy any file outside of this directory. For example if the context is app/ then you cannot copy anything from ../app
Check .dockerignore to be sure that you are not ignoring the file you want to copy.

edited Oct 19 '20 at 02:37

answered Oct 17 '20 at 16:03

Ash Singh

3,921
2
25
30

score 0 · Answer 5 · answered Feb 03 '21 at 12:50

I got it working by first checking what the context was, setting an absolute path before the source file in your Dockerfile to get that information:

# grep COPY Dockerfile
COPY /path/to/foo   /whatever/path/in/destination/foo

Building with that:

docker build -t bar/foo .

you'll get an error, which states the context-path that Docker is apparently looking into for its files, e.g. it turns out to be:

/var/lib/docker/tmp      # I don't remember

Copying(!) your set of build-files in that directory (here: /var/lib/docker/tmp), cd into it, build from there.

See if that works, and don't forget to do some housekeeping cleaning up the tmp, deleting your files before the next visit(or).

HTH

Michael

score 0 · Answer 6 · answered Oct 10 '22 at 20:56

Got this error using a Dockerfile for a linux container on a Windows machine:

#24 1.160 Skipping project "/src/Common/MetaData/Metadata.csproj" because it was not found.

Restore worked perfectly on the host machine.

Turned out to be the error mentioned here:

https://stackoverflow.com/a/68592423/3850405

A .csproj file did not match casing in Visual Studio vs the file system.

score 0 · Answer 7 · answered Apr 20 '23 at 14:50

In addition to correct context, there is the semantic difference between Dockerfile COPY and bash cp command.

Example in bash:

~: mkdir -p a/b
~: mkdir c
~: touch a/file1 a/b/file2
~: cp -R a c  # Copying directory 'a' to 'c', recursively
~: ls -lR c # Listing directory 'c', recursively
c:
total 0
drwxr-xr-x 4 teixeira staff 128 Abr 20 11:08 a

c/a:
total 0
drwxr-xr-x 3 teixeira staff 96 Abr 20 11:08 b
-rw-r--r-- 1 teixeira staff  0 Abr 20 11:08 file1

c/a/b:
total 0
-rw-r--r-- 1 teixeira staff 0 Abr 20 11:08 file2

Dockerfile.teste

FROM debian:bullseye-slim

RUN mkdir c
WORKDIR c

# Here's the issue, you will see soon why
COPY ./a . # Copying directory 'a' to 'c', recursively

RUN ls -lR . # listing directory 'c', recursively

docker-compose.yml

version: '3.8'
services:
  teste:
    image: teste
    build:
      # My local context is correct, just for completeness  
      context: $RAIZ_PROJETO
      dockerfile: docker/Dockerfile.teste

.env

RAIZ_PROJETO=/home/vagrant/docker/testes

Now, the output of building the above image:

~: sudo docker compose build --progress plain --no-cache teste

(...)
#6 [2/5] RUN mkdir c
#6 DONE 0.4s

#7 [3/5] WORKDIR c
#7 DONE 0.0s

#8 [4/5] COPY ./a .
#8 DONE 0.0s

#9 [5/5] RUN ls -lR .
#9 0.306 .:
#9 0.306 total 4
#9 0.306 drwxr-xr-x 2 root root 4096 Apr 20 14:33 b
#9 0.306 -rw-r--r-- 1 root root    0 Apr 20 14:13 file1
#9 0.306
#9 0.306 ./b:
#9 0.306 total 0
#9 0.306 -rw-r--r-- 1 root root 0 Apr 20 14:13 file2
#9 DONE 0.3s
(...)

Did you notice the difference ? The directory a content was copied to directory c in the docker image, and not a itself!

TL;DR;

When you copy a directory to a destination in bash, the directory itself is copied to the destination. That differs from COPY, in which if you want to copy the directory itself, you must specify its name again in the destination.

So, to fix the above Dockerfile, simply repeat the directory source to the COPY destination:

COPY ./a ./a

Correct output of docker compose build

#9 [5/5] RUN ls -lR .
#9 0.341 .:
#9 0.341 total 4
#9 0.341 drwxr-xr-x 3 root root 4096 Apr 20 14:48 a
#9 0.341
#9 0.341 ./a:
#9 0.341 total 4
#9 0.341 drwxr-xr-x 2 root root 4096 Apr 20 14:33 b
#9 0.341 -rw-r--r-- 1 root root    0 Apr 20 14:13 file1
#9 0.341
#9 0.341 ./a/b:
#9 0.341 total 0
#9 0.341 -rw-r--r-- 1 root root 0 Apr 20 14:13 file2
#9 DONE 0.4s

Dockerfile COPY instruction failing?

7 Answers7

Linked