0

I have a list of files in a file called FileList.txt. The are about 120 files in this text file all ending with the same suffix .fastq.gz. An example of the first 6 rows of text:

sample1_R1.fastq.gz
sample1_R2.fastq.gz
sample2_R1.fastq.gz
sample2_R2.fastq.gz
sample3_R1.fastq.gz
sample3_R2.fastq.gz

These files live within the following folder structure: Completed_files\testcomplete_041323\041323. The last part of the last folder where the files live is the date (MMDDYY). The layer of complexity is that the files live in different-dated folders (i.e., 4 files from FileList.txt live in testcomplete_041323\041323, 6 files live in testcomplete_040523\040523 and so on...). I need a Windows batch file that copies all of the files from FileList.txt into a different directory. Here is the current batch script I have:

@ECHO off
SET FileListTxt=FileList.txt
SET SourceFolder="testcomplete*"
SET DestinationFolder="out_folder"

for /f "tokens=* delims=" %%a in ('type "%FileListTxt%"') do robocopy "%SourceFolder%" "%DestinationFolder%" "%%a"

ECHO DONE COPYING FILES IN %FileListTxt% FROM %SourceFolder% TO %DestinationFolder%

I put FileList.txt and this batch script in Completed_files and ran the script. The issue I am observing with this code is that it is creating empty DIRECTORIES named with each of these files and not copying the files themselves.

R.T. Canterbury
  • 50
  • 6
  • `%SystemRoot%\System32\robocopy.exe` is for copying/moving many files/directories from one folder to another folder as it can be read on running in a command prompt window `robocopy /?` or reading the Microsoft documentation for [robocopy](https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/robocopy) or the SS64 documentation for [ROBOCOPY](https://ss64.com/nt/robocopy.html). The source folder path __cannot__ contain a wildcard character like `*`. The usage of a __FOR__ loop is not necessary at all on using __ROBOCOPY__ as designed for. – Mofi Apr 19 '23 at 20:02
  • Please read my answer on [Why is no string output with 'echo %var%' after using 'set var = text' on command line?](https://stackoverflow.com/a/26388460/3074564) It explains the big difference between `SET DestinationFolder="out_folder"` with first `"` right to equal sign and `SET "DestinationFolder=out_folder"` with first `"` left to variable name. Then look with [debugging your batch file](https://stackoverflow.com/a/42448601/3074564) how `robocopy` would be executed by our batch file. Do you see what is wrong on `""out_folder""` as destination folder path argument string? – Mofi Apr 19 '23 at 20:07
  • Next run in the now already opened command prompt window `for /?` and read the output help carefully from top of first to bottom of last page. `for /F` can be used to run in background `%ComSpec% /c type "FileList.txt"`, capture the output and process the captured output line by line. But why not using `for /F` to directly process the lines in text file `FileList.txt` without running one more `%SystemRoot%\System32\cmd.exe` to run its internal command `type` to output the lines of a text file? Is the file `FileList.txt` a Unicode file with the characters encoded with UTF-16 Little Endian? – Mofi Apr 19 '23 at 20:12
  • What is not clear for me is how the folder `out_folder` should look like after batch file execution. Should all files in `FileList.txt` copied into `out_folder` which finally contains 120 files or should the files be copied with replication of the source folder paths after ``Completed_files\`` in ``out_folder\``. A source directory tree with files and the expected destination directory tree with files would be very helpful to understand the task completely. – Mofi Apr 19 '23 at 20:17
  • 1
    There are multiple solutions possible for copying all files in the list file just into the directory `out_folder`. An efficient method would be `md "out_folder" 2^>nul` to create the folder on not already existing in __current directory__ (can be different to batch file directory) and use next `if exist "out_folder\" for /F "delims=" %%I in ('dir "Completed_files\*.fastq.gz /A-D-L /B /S 2^>nul ^| %SystemRoot%\System32\findstr.exe /E /I /L /G:"FileList.txt"') do copy /B "%%I" "out_folder\" >nul` – Mofi Apr 19 '23 at 20:25
  • That is not a perfect solution. The text file `FileList.txt` could contain `sample1_R2.fastq.gz` and there is perhaps a file with name `Another_sample1_R2.fastq.gz` which is also copied by the command line above. But if such partial file names do not exist in your directory tree, the above command line is very efficient to copy all the files in the file names list into the directory `out_folder` without replicating the folder structure. – Mofi Apr 19 '23 at 20:27
  • Another method would be: `if exist "out_folder\" for /F "usebackq eol=| delims=" %%I in ("FileList.txt") do for /D %%J in ("Completed_files\testcomplete*") do if exist "%%J\%%I" copy "%%J\%%I" "out_folder\" >nul` It is less efficient as the above command line, but it avoids copying a file of which file name contains just at end of complete file name a file name in the list file. One more solution would be: `if exist "out_folder\" for /D %%J in ("Completed_files\testcomplete*") do for /F "usebackq eol=| delims=" %%I in ("FileList.txt") do if exist "%%J\%%I" copy "%%J\%%I" "out_folder\" >nul` – Mofi Apr 19 '23 at 20:44
  • @Mofi - thank you for the extensive explanation for the strategy behind this and for offering multiple situations/paths to solve this. Your comment that I upvoted was the answer I needed because all of the files are named with the same convention (no other partial names on any line) and was extremely efficient. I think your last comment is the most robust approach though for more general file copying if you have files that will be have different prefixes in the names. Thank you again. – R.T. Canterbury Apr 25 '23 at 17:17

0 Answers0