Simple Bash File Search and Copy

Question

I'm actually new to bash scripting. I'm trying to create a script that will read a text file containing a list of file "root" names (example: GBD_22, TBDDR_04, etc.). I need to find each file that has names that begin with these roots (example: GBD_22.R1.fastq.gz, GBD_22.R2.fastq.gz, GBD_22.read.gz, TBDDR_04.R1.fastq.gz, and so forth). The files scattered in a series of subfolders inside a main "source" directory. I need to find and copy the files from wherever they are to a main "output" directory also in the pwd. The text file, the source directory and the output directory are all at the same level in the pwd. So far I have started with the following script:

#!/bin/env bash
for file in $(cat ~/name.txt)
do cp "$file"* ~/OUTPUTDIR
done

The script reads the text file with the list of file "family" names, but fails to actually do the file find in the "deposit" directory, much less copy the files. Any help is very much appreciated. When I run the script, the result is:

  cp: cannot stat 'CBD_22*': No such file or directory
  cp: cannot stat 'CBD_23*': No such file or directory
  cp: cannot stat 'TBDDR_04*': No such file or directory

score -1 · Answer 1 · answered Nov 01 '21 at 17:53

-1

It is not clear how deep is the 'series of subfolders', so the script below employs find to locate all the files in a given directory and its subdirectories and copies them to a destination path creating a directory for each family.

It is supposed that INPUT_DIR and OUTPUT_DIR do not overlap as do family names. If a family name contains dot and another family name is its substring, the result may be weird.

#!/usr/bin/env bash

INPUT_DIR=./src
OUTPUT_DIR=./dest
FAMILY_NAMES=names

while IFS=$'\n' read -r family; do
    dest="$OUTPUT_DIR/$family"
    mkdir -p "$dest"
    find "$INPUT_DIR" -name "$family.*" -exec cp '{}' "$dest/" \;
    # remove empty dirs
    rmdir "$dest" 2>/dev/null
done < "$FAMILY_NAMES"

answered Nov 01 '21 at 17:53

gluk47

1,812
20
31

This is not working. Let me see if can explain better. I have a number of files scattered among several subfolders inside a main "src" folder. Outside of this "src" and in the same home directory I have a text file containing a list of the root names of the files (e.g. abc, def, etc). I need to locate the files with these roots (e.g., abc_1.R2.fastq, def_2.R1.fastq). I need to copy these files to general "dest" folder located at the same level of the main level of the "src" folder. "dest", "src" and names.txt are in the same working directory. I'm confused why your script has a mkdir – Lou_A Nov 02 '21 at 14:41
Also, I checked the question to which I was linked and it does not answer my question. – Lou_A Nov 02 '21 at 14:42
When I run your script, I get the following errors: cp: cannot create regular file '/CG17N_33/': Not a directory cp: cannot create regular file '/CG17N_33/': Not a directory cp: cannot create regular file '/CG17N_34/': Not a directory cp: cannot create regular file '/CG17N_34/': Not a directory cp: cannot create regular file '/CG17N_35/': Not a directory cp: cannot create regular file '/CG17N_35/': Not a directory – Lou_A Nov 02 '21 at 14:53
In addition, and in this particular case at least, the name of the subdirectories match the root name of the files that are listed in "names"; however that is not always going to be the case. Sometimes the subdirectory will not match the root of the file name. – Lou_A Nov 02 '21 at 14:58

Simple Bash File Search and Copy

1 Answers1