How to copy multiple files from container to host using docker cp

Question

I want to use wildcard to select multiple files from a directory in a container and use docker cp to copy these files from container to docker host.

I couldn't find if support for using wildcard is available with docker cp yet or not.

docker cp fd87af99b650:/foo/metrics.csv* /root/metrices_testing/

This results with the error metrics.csv*: no such file or directory

I came across an example where for loop was used to select a few files and then sent to container, but i want to transfer files from container to host and want to do this on docker host itself as script is running on host only.

Using docker exec to select files first and then copying them using docker cp can be an option. But that is a 2 step process.

Can someone please help me do this in one step?

EDIT:

I tried this. A step close but still failing.

# for f in $(docker exec -it SPSRS bash -c "ls /opt/tpa/logs/metrics.csv*");
     do docker cp SPSRS:$f /root/metrices_testing/;
 done


: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-08:45
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-09:00
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-09:15
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-09:30
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-09:45
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-10:00
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-10:15
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-10:30
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-10:45
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-11:00
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-11:15
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-11:30
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-11:45
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-12:00
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-12:15
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-12:30
: no such file or directory lstat /docker/overlay2/193d2ad0d8d087377e3b96cbfb672b0e39132ae5e961872127614c9396f8c068/merged/opt/SPS_18_5_R1/logs/metrics.csv.2018.07.10-12:45

atline · Accepted Answer · 2022-11-11T05:26:55.777

7

In fact your solution can make your aims just need a little change:

for f in $(docker exec -it SPSRS bash -c "ls /opt/tpa/logs/metrics.csv*"); do docker cp SPSRS:$f /root/metrices_testing/; done

->

for f in $(docker exec SPSRS bash -c "ls /opt/tpa/logs/metrics.csv*"); do docker cp SPSRS:`echo $f | sed 's/\r//g'` /root/metrices_testing/; done

This is because docker exec SPSRS bash -c "ls /opt/tpa/logs/metrics.csv*" will have \r in every matched string, so finally the cp can not find the files in container.

So, we use echo $f | sed 's/\r//g' to get rid of \r for every file name, this could make you work.

NOTE: for alpine, we need to use sh to replace bash, meanwhile, -it should be deleted to avoid colorful print in alpine introduce some invisible characters like ^[[0;0m, etc.

edited Nov 11 '22 at 05:26

answered Jul 10 '18 at 15:36

atline

28,355
16
77
113

it lists the files with the full paths but still shows no files, may it not work in the alpine image? @atline – AATHITH RAJENDRAN Nov 10 '22 at 05:14
@AATHITHRAJENDRAN Usually, alpine image doesn't have `bash`, so you can change it to `sh -c` instead. – atline Nov 10 '22 at 07:21
yes I tried that before posting, didn't worked – AATHITH RAJENDRAN Nov 10 '22 at 07:34
node-14-alpine, it has a sh – AATHITH RAJENDRAN Nov 10 '22 at 14:34
@AATHITHRAJENDRAN You are correct, I did wrong experiment for alpine. It looks in alpine image, it enables "colorful print" for `ls`, means every line it will have something like `^[[0;0m` control character in print which not visible, but really affect the program. I updated the answer to remove `-it`, now it works both for alpine and non-alpine. – atline Nov 11 '22 at 05:26

score 6 · Answer 2 · answered Apr 07 '20 at 17:56

6

Docker cp command supports to copy folder with all the contents inside a folder

docker cp -a container-id:/opt/tpa/logs/  /root/testing/

In the above example copying files from container folder /opt/tpa/logs to local machine /root/testing/ folder. Here all the files inside /logs/ will be copied to local. The trick here is using -a option along with docker cp

answered Apr 07 '20 at 17:56

Praveen

106
1
4

1

What's the trick with `-a` you are referring to? – not2savvy Jul 16 '21 at 10:45
this doesn't work, it will create `logs/` dir inside `/root/testing` like this `/root/testing/logs/` then the files will be present inside this. `--archive , -a Archive mode (copy all uid/gid information)` – AATHITH RAJENDRAN Nov 10 '22 at 04:26

score 2 · Answer 3 · answered Jul 10 '18 at 12:31

2

Docker cp still doesn't support wildcards. You can however use them in a Dockerfile in the following way:

COPY hom* /mydir/        # adds all files starting with "hom"
COPY hom?.txt /mydir/    # ? is replaced with any single character, e.g., "home.txt"

Reference: https://docs.docker.com/engine/reference/builder/#copy

answered Jul 10 '18 at 12:31

Neekoy

2,325
5
29
48

Thanks for the response. But i can't use Dockerfile for this. I've mapped the folder as a volume and will be copying the metrics on runtime. For some access constraints i need to copy it to docker host. – Lucky Tyagi Jul 10 '18 at 12:40
Hmm, wouldn't it be more convenient to have a bind volume between a folder on the host and a folder in the container so they are synced, and you won't need to manually copy the files? – Neekoy Jul 10 '18 at 13:01

score 1 · Answer 4 · answered Jul 02 '20 at 03:35

Run this inside the container:

dcp() {
  if [ "$#" -eq 1 ]; then
    printf "docker cp %q .\n" "$(hostname):$(readlink -e "$1")"
  else
    local archive="$(mktemp -t "export-XXXXX.tgz")"
    tar czf "$archive" "$@" --checkpoint=.52428800
    printf "docker exec %q cat %q | tar xvz -C .\n" "$(hostname)" "$archive"
  fi
}

Then select the files you want to copy out:

dcp /foo/metrics.csv*

It'll create an archive inside of the container and spit out a command for you to run. Run that command on the host.

e.g.

docker exec 1c75ed99fa42 cat /tmp/export-x9hg6.tgz | tar xvz -C .

Or, I guess you could do it without the temporary archive:

dcp() {
  if [ "$#" -eq 1 ]; then
    printf "docker cp %q .\n" "$(hostname):$(readlink -e "$1")"
  else
    printf "docker exec %q tar czC %q" "$(hostname)" "$PWD"
    printf " %q" "$@"
    printf " | tar xzvC .\n"
  fi
}

Will generate a command for you, like:

docker exec 1c75ed99fa42 tar czC /root .cache .zcompdump .zinit .zshrc .zshrc.d foo\ bar | tar xzvC .

You don't even need the alias then, it's just a convenience.

score 0 · Answer 5 · answered Sep 29 '20 at 07:46

docker cp accepts either files, or tar archives, so you can pack the list of files provided as arguments to an tar archive, return the archive to stdout and pipe to docker cp.

#!/bin/bash

if [[ "$#" -lt 2 || "$1" == "-h" || "$1" == "--help" ]]; then
    printf "Copy files to docker container directory.\n\n"
    echo "Usage: $(basename $0) files... container:directory"
    exit 0
fi

SOURCE="${*%${!#}}"
TARGET="${@:$#}"

tar cf - $SOURCE | docker cp - $TARGET

score 0 · Answer 6 · answered Feb 28 '23 at 09:34

Another one-liner: First get a list of the desired files with ls inside the container, which works with wildcards. Then pass each line of ls as part of the argument to docker cp in a while loop:

In your case:

docker exec container bash -c "ls foo/metrics.csv*" | while read line; do docker cp container:/$line /root/metrices_testing/; done

How to copy multiple files from container to host using docker cp

6 Answers6