1

Been trying for hours to get this to work - but I'm a complete novice (e.g. started using terminal today) and struggling to get this working.

I have a csv file that lists filenames row by row. I want to find them and copy them to a new directory. The files are listed in one directory, but each file is in its own subdirectory.

This is what I have tried, but it is not copying anything. But I can see it cycling through the searches:

    cat /Users/Will/Downloads/three-new.csv | while IFS=, read col1
do
    find /Volumes/LaCie\ 8tb/BPOTY/2021\ Photos -path $col1 -exec cp {} /Volumes/LaCie\ 8tb/BPOTY/2021\ book \;
done

I have also tried this, but it just copies all files from the directory (stripping the subdirectories) and doesn't find the actual filenames.

cat /Users/Will/Downloads/three-new.csv | while IFS=, read col1
do
    find /Volumes/LaCie\ 8tb/BPOTY/2021\ Photos $col1 -exec cp {} /Volumes/LaCie\ 8tb/BPOTY/2021\ book \;
done

The file can be seen here: https://www.dropbox.com/s/6uqwzky43r25ors/three.csv?dl=0

Have run a command as requested and the CSV outputs:

00000000: 636f 6c31 0d0a 494d 475f 3437 3538 2d64  col1..IMG_4758-d
00000010: 6e2e 6a70 670d 0a43 4f45 5f32 3330 352d  n.jpg..COE_2305-
00000020: 4564 6974 2e74 6966 2e7a 6970 0d0a 5f44  Edit.tif.zip.._D
00000030: 5343 3435 3933 2d68 6972 6573 2d72 6573  SC4593-hires-res
00000040: 6f6c 7574 696f 6e2d 5065 7465 722d 6365  olution-Peter-ce
00000050: 6368 2d54 6865 2d6c 6f6f 6b2e 6a70 67    ch-The-look.jpg
  • You need to include a representative sample of the **CSV** _file_. – user3439894 Mar 15 '21 at 00:54
  • I see at least two problems, though there isn't enough information here to tell if they're what's making it fail. First, `read col1` will put entire lines (not just values from the first column) into `col1` (because `read` puts any extra fields in the last variable), so use `read col1 ignored` (and then don't use the `ignored` variable). Second, you need to double-quote variable references, i.e. `"$col1"` instead of just `$col1`. [Shellcheck.net](https://www.shellcheck.net) is good at pointing out common mistakes like this, and I strongly recommend it. – Gordon Davisson Mar 15 '21 at 00:55
  • @user3439894 https://www.dropbox.com/s/6uqwzky43r25ors/three.csv?dl=0 – Will Nicholls Mar 15 '21 at 00:58
  • @GordonDavisson Thanks. I have tried this but the command running in the terminal top bar then shows -name ?? instead of the actual filename. – Will Nicholls Mar 15 '21 at 00:59
  • Put `set -x` at the beginning of the script, so it'll print a trace of what it's doing. Oh, and while you're debugging it'd be safer to make it just `echo` the `cp` commands instead of (trying to) run them -- that is, replace `... -exec cp ...` with `... -exec echo cp ...` – Gordon Davisson Mar 15 '21 at 01:03
  • @GordonDavisson it comes back with`+ IFS=, + read col1 ignored + set -x find '/Volumes/LaCie 8tb/BPOTY/2021 Photos' -path $'?\200\234\200\235' -exec echo cp '{}' '/Volumes/LaCie 8tb/BPOTY/2021 book' ';' + IFS=, + read col1 ignored + set -x find '/Volumes/LaCie 8tb/BPOTY/2021 Photos' -path $'?\200\234\200\235' -exec echo cp '{}' '/Volumes/LaCie 8tb/BPOTY/2021 book' ';' + IFS=, + read col1 ignored ` – Will Nicholls Mar 15 '21 at 01:05
  • There's something really weird with that csv file. The one you put on Dropbox is in DOS/Windows format, which is a problem (see [this question](https://stackoverflow.com/questions/39527571/are-shell-scripts-sensitive-to-encoding-and-line-endings)), but that trace indicates completely different. Try viewing the file's contents with `xxd /Users/Will/Downloads/three-new.csv` and edit the results into your question. – Gordon Davisson Mar 15 '21 at 01:10
  • @GordonDavisson Looks like you're right. Added that into the question. – Will Nicholls Mar 15 '21 at 01:12
  • @GordonDavisson I put the values into a google sheet and exported as .csv -- same result. – Will Nicholls Mar 15 '21 at 01:14
  • In **Terminal**: `tr -d '\r' < three.csv > new.csv` to create a new _file_ without the _carriage returns_. – user3439894 Mar 15 '21 at 01:18
  • @user3439894 have run that (with the paths) and then tried running the script on the new file: `+ IFS=, + read col1 + set -x find '/Volumes/LaCie 8tb/BPOTY/2021 Photos' -path $'?\200\234\200\235' -exec echo cp '{}' '/Volumes/LaCie 8tb/BPOTY/2021 book' ';' + IFS=, + read col1` – Will Nicholls Mar 15 '21 at 01:20
  • So to be clear, there is only one _column_ in the **CSV** _file_ and there are actually no _commas_ and the entries are not quoted, correct? – user3439894 Mar 15 '21 at 01:24
  • @user3439894 correct – Will Nicholls Mar 15 '21 at 01:26
  • @user3439894 It runs but says "-bash: cd: /Volumes/LaCie\ 8tb/BPOTY/2021\ Photos: No such file or directory". I wonder if this is because the files are each within their own subdirectory one level deeper? – Will Nicholls Mar 15 '21 at 01:38
  • Do not escape the _spaces_ with a `\\`, just _quote_ the _paths_. – user3439894 Mar 15 '21 at 01:41
  • @user3439894 Tried that after and it says: `find: -f: unknown primary or operator find: -f: unknown primary or operator` – Will Nicholls Mar 15 '21 at 01:43
  • Sorry, that was a typo: `find . -type f -iname "$line" -exec cp -av {} "/path/to/destnation_dir" \;` – user3439894 Mar 15 '21 at 01:44
  • @user3439894 it copied the first two, but not the third. progress! weirdly, I tried it again (after removing the copied files) and it didn't repeat. For the life of me can't get it to happen again successfully. – Will Nicholls Mar 15 '21 at 01:49
  • It didn't copy the third line because there is no _linefeed_ at the end of it. – user3439894 Mar 15 '21 at 01:54
  • @user3439894 What do you mean by line feed please? I can't get it to repeat for some reason. – Will Nicholls Mar 15 '21 at 01:54
  • Whenever I rerun the script you gave me, nothing happens and then command just hangs. – Will Nicholls Mar 15 '21 at 02:01
  • The third line of the **CSV** _file_ does not have a _line feed_ after it. Open the file, place the cursor at the end of the third line and press _enter_ and save the file. – user3439894 Mar 15 '21 at 02:13
  • BTW I just tested the _code_ I posted after correcting it as shown in the previous comment and it works. – user3439894 Mar 15 '21 at 02:14
  • @user3439894 Ok - that's fine. However the problem now is the script just won't run anymore. I'm putting exactly the same thing in. I've tried exiting, reopening terminal and running it again but it just gives me two audible "ding" noises and nothing happens. Just stays with > on the command line. – Will Nicholls Mar 15 '21 at 02:15
  • @user3439894 This is what I'm running - which initially did work. But it's now just hanging: `#!/bin/bash while IFS= read -r line; do cd "/Volumes/LaCie 8tb/BPOTY/2021 Photos" || exit find . -type f -iname "$line" -exec cp -av {} "/Volumes/LaCie 8tb/BPOTY/2021 book" \; done < "/Users/Will/Downloads/new.csv”` – Will Nicholls Mar 15 '21 at 02:16
  • The closing _double-quote_ is bad. – user3439894 Mar 15 '21 at 02:19
  • @user3439894 AMAZING! That was it. My friend, I can't thank you enough for helping me. This has been such a stressful evening as this task is necessary for something I have a deadline for tomorrow. THANK YOU. – Will Nicholls Mar 15 '21 at 02:21

1 Answers1

0

Since your are doing this under macOS and the file has CRLF endings, first lets remove the carriage returns from the three.csv file, in Terminal:

tr -d '\r' < three.csv > new.csv

As the three.csv file only has one column and there are actually no commas, nor are the entries quoted, this is really not a CSV file and can be treated in a more straight forward manner. To that end, use the following shell script syntax to accomplish your goal.

#!/bin/bash

while IFS= read -r line; do
    cd '/path/to/parent_dir' || exit
    find . -type f -iname "$line" -exec cp -av {} '/path/to/destnation_dir' \;
done < '/path/to/new.csv'


Notes:

  • The cd '/path/to/parent_dir' || exit isn't absolutely necessary, it's just my preferred method in many use cases. The . after find can be replaced with '/path/to/parent_directory'.
  • Pathnames with spaces need to either be escaped with a backslash character or the pathname be quoted, single, or double if containing variables, however, not both. As a general rule I almost alway quote non-variable strings with single-quotes. (I did use double-quotes last night on non-variable strings but that was a typo.)
  • If executing directly in Terminal then the #!/bin/bash shebang isn't necessary and was included if using it as a standalone shell script.
  • The last line of the three.csv file, and thus the new.csv file does not have a line feed character. Edit the file by placing the cursor at the end of line three and press enter, then save the file. Otherwise the last line is not processed.
  • As mentioned in the comments of the OP by Gordon Davisson, quote your variables and use ShellCheck to check your bash shell script code.
user3439894
  • 7,266
  • 3
  • 17
  • 28