1

I have a bash script that reads a csv file line by line.

After every reading each line it will execute docker cp command.

However when it is executed the script fails to copy the files in the container.

INPUT=FileIdsToRemove.csv
COPYDIR=/var/lib/mayan/document_file_storage
OLDIFS=$IFS
IFS=','
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read fileuuid
do
        echo "uuid : $fileuuid";
        docker cp mayan_app_1:/var/lib/mayan/document_file_storage/"$fileuuid" ./;

done < $INPUT
IFS=$OLDIFS

output:

uuid : 87d46881-0839-4ae1-aa20-3a056c2848c1
Error: No such container:path: mayan_app_1:/var/lib/mayan/document_file_storage/87d46881-0839-4ae1-aa20-3a056c2848c1

I am able to copy each file when I execute docker cp in the terminal instead of executing the script.

docker cp mayan_app_1:/var/lib/mayan/document_file_storage/0fd5196e-fd01-44f7-91dc-ef1c9c5408f4 ./

Below is the csv file content for reference

fileuuid
c61bd18a-47ef-448e-a04c-fd95b40d70c5
79c473e3-2f16-444e-9fe7-e02ce53f046f
87d46881-0839-4ae1-aa20-3a056c2848c1
16b1dc1f-e10f-4681-896a-d8fb3c421d22
792b363c-fa23-4f07-85a6-daa33ec4c5bd
jalwan
  • 55
  • 10
  • The script will read the fileuuid and insert it into the end of the path /var/lib/mayan/document_file_storage/"$fileuuid" I tried like this too "/var/lib/mayan/document_file_storage/$fileuuid" and it did not work. – jalwan May 05 '23 at 07:34
  • All we can conclude is what the error message already says; there is no file with that name in the container. – tripleee May 05 '23 at 07:36
  • 1
    As an aside, you'll want to paste your script at https://shellcheck.net/ and fix the many beginner errors. [Quote your variables](https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable) and [don't use upper case for your private variables.](https://stackoverflow.com/questions/673055/correct-bash-and-shell-script-variable-capitalization) – tripleee May 05 '23 at 07:37
  • For debugging, I would as a first step do a `docker ls -l mayan_app_1:/var/lib/mayan/document_file_storage` before the copying; this would - among others - verify that the parent directory of the file is accessible. – user1934428 May 05 '23 at 07:57
  • As a next step, I would paste the `docker cp` command, which you claim that it works in your interactive shell, literally into your script (perhaps as the first line). Finally, I would print the `uuid` not with `echo`, but with `printf %s $fileuuid||xxd`, to see whether you have some weird characters. In particular, look out 0d characters. – user1934428 May 05 '23 at 08:05
  • Thank you i will try the debugging and let you know – jalwan May 05 '23 at 08:49
  • A possible cause of the problem is that the `FileIdsToRemove.csv` file has Windows line termination (CR-LF) instead of Unix/Linux line termination (LF). See [Are shell scripts sensitive to encoding and line endings?](https://stackoverflow.com/q/39527571/4154375) and [Why does my tool output overwrite itself and how do I fix it?](https://stackoverflow.com/q/45772525/4154375). Either change the line termination in the file or modify the shell code to remove the CR (e.g. `fileuuid=${fileuuid%$'\r'}`). – pjh May 05 '23 at 12:11
  • The code for saving and restoring `IFS` doesn't work in general. See [Is it a sane approach to "back up" the $IFS variable?](https://unix.stackexchange.com/q/264926/264812). A much better option in this case is to replace `while read fileuuid` with `while IFS=, read -r fileuuid`. See [BashFAQ/001 (How can I read a file \(data stream, variable\) line-by-line \(and/or field-by-field)?)](https://mywiki.wooledge.org/BashFAQ/001). – pjh May 05 '23 at 12:17

1 Answers1

0

The script now works. I summarized how it works below.

read csv file line by line run $(echo "$fileuuid" | sed 's/\r$//' | tr -d '\n')"; and store the output in a variable, i think this is the filename without \r character.

run docker cp mayan_app_1:/var/lib/mayan/document_file_storage/"$fileuuidclean" ./; after cleaning the filename to copy the file into the folder.

Below is the new script.

INPUT=FileIdsToRemove.csv
COPYDIR=/var/lib/mayan/document_file_storage
OLDIFS=$IFS
IFS=','
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read fileuuid
do
        fileuuidclean="$(echo "$fileuuid" | sed 's/\r$//' | tr -d '\n')";
        docker cp mayan_app_1:/var/lib/mayan/document_file_storage/"$fileuuidclean" ./;

done < $INPUT
IFS=$OLDIFS
jalwan
  • 55
  • 10