append folder name to file name when copying-space in folder name

Question

I have been using a loop to copy data from an external group to my working directory, appending the folder name to the beginning of the file name. The for loop is based on this previous question. Append part of folder name to all .gz within

Since I asked that question the external group simplified their folder structure, but now has added " (2)" to some of the folder names (I have no ability to influence the way that the group names their files, it's a giant company). The space has broken my loop and I need help fixing it.

The file structure on the external group

  Samples/SampleName1/Files/SampleID1_uniqueNumber.gz
  Samples/SampleName2/Files/SampleID2_uniqueNumber.gz
  Samples/SampleName3/Files/SampleID3_uniqueNumber.gz
  Samples/SampleName3 (2)/Files/SampleID3_uniqueNumber.gz

What I want within my destination folder (all samples moved to single destination)

  SampleName1.SampleID1_uniqueNumber.gz
  SampleName2.SampleID2_uniqueNumber.gz
  SampleName3.SampleID3_uniqueNumber.gz
  SampleName3.SampleID3_uniqueNumber.gz

My current for loop which correctly copies everything except for the last sample. The uniqueNumber should be unique enough to prevent the 2 SampleName3 from overwriting each other.

   for f in ../pathToData/Samples/*/Files/*.gz;
    do s=${f##../pathToData/Samples/}; 
    s=${s%%/*};
    cp $f "/destinationFolder/"$s"."${f##*Files/};
    done

How do I escape out the space so cp sees " (2)" as part of the original file name not "(2)" as the destination folder?

score 2 · Answer 1 · answered Mar 27 '18 at 21:13

2

Unlike some programming languages, bash can do variable replacement the within quotes. To pass a variable as a single argument, surround it in quotes:

cp "$f" [...]

This also applies to the rest of the command. It can be rewritten as:

cp "$f" "/destinationFolder/$s.${f##*Files/}";

answered Mar 27 '18 at 21:13

Azsgy

3,139
2
29
40

seems too simple, didn't occur to me that parameter expansion works within quotes. Thanks! – thermophile Mar 28 '18 at 15:40
This is almost it, I need to strip out the " (2)" in the destination file name. – thermophile Mar 28 '18 at 22:43
That is a seperate question, so you should re-mark this question as answered and open a new one. Feel free to link back to this question in your new question though. – Azsgy Mar 28 '18 at 22:48

score 1 · Answer 2 · edited Mar 16 '22 at 15:35

If you want to do it your way :

for f in Samples/*/Files/*.gz; do
    s=${f##Samples/};
    s=${s%%/*};
    [[ $s =~ [[:space:]] ]] && continue
    echo cp "$f" "/destinationFolder/$s.${f##*Files/}"
done

(remove echo command when it looks good enough)

But you don't need any loop, please, take a look :

Input :

(you have a dupe file : SampleName3.SampleID3_uniqueNumber.gz)

$ tree Samples/
Samples/
├── SampleName1
│   └── Files
│       └── SampleID1_uniqueNumber.gz
├── SampleName2
│   └── Files
│       └── SampleID2_uniqueNumber.gz
├── SampleName3
│   └── Files
│       └── SampleID3_uniqueNumber.gz
└── SampleName3 (2)
    └── Files
        └── SampleID3_uniqueNumber.gz

8 directories, 4 files

Code using perl's rename in a bash4 shell :

$ shopt -s globstar # enable recursion with '**' require bash --version >= 4
$ rename -n 's|^[^/]+/([^/\s]+)/Files/(.*\.gz)|$1.$2|' Samples/**/*.gz

Output :

Samples/SampleName1/Files/SampleID1_uniqueNumber.gz -> SampleName1.SampleID1_uniqueNumber.gz
Samples/SampleName2/Files/SampleID2_uniqueNumber.gz -> SampleName2.SampleID2_uniqueNumber.gz
Samples/SampleName3/Files/SampleID3_uniqueNumber.gz -> SampleName3.SampleID3_uniqueNumber.gz

Remove -n switch when the output looks good.

There are other tools with the same name which may or may not be able to do this, so be careful.

If you run the following command (GNU)

$ file "$(readlink -f "$(type -p rename)")"

and you have a result that contains Perl script, ASCII text executable and not containing ELF, then this seems to be the right tool =)

If not, to make it the default (usually already the case) on Debian and derivative like Ubuntu :

$ sudo update-alternatives --set rename /path/to/rename

Replace /path/to/rename to the path of your perl rename executable.

If you don't have this command, search your package manager to install it or do it manually (no deps...)

_{^{This tool was originally written by Larry Wall, the Perl's dad.}}

For this task, I'd prefer to use tools that don't need updating or special installation as I'm trying to write simple commands that can be used by people with very little experience in command line (information that wasn't in my question). But thanks for the example. — thermophile, Mar 28 '18 at 15:48

append folder name to file name when copying-space in folder name

2 Answers2

Input :

Code using perl's rename in a bash4 shell :

Output :