0

I'm trying to retrieve the paths to files and save those paths to a text file. Only the last iteration of the loop seems to be working. I've tried in both csh and in bash (making sure to change the shell environment as appropriate). Here is the bash script:

#!/bin/bash

#CSV file containing the list of strings to search for

strings_file="/project/crism/users/phillms1/Murchie2017MDAP/tyrrhena_tile_ids.csv"

#Output file where the full path of the file containing the strings will be appended

output_file="/project/crism/users/phillms1/Murchie2017MDAP/tyrrhena_tile_id_paths.txt"

#Read the strings from the CSV file

strings=$(cat "$strings_file")

#Search the current directory and all subdirectories for files containing the strings and "MRRSU" in their filename

for str in $strings; do
find . -type f -name "*$str*MRRSU*.IMG" -exec readlink -f {} ; >> "$output_file"
done

and here is the csh script:

#!/bin/csh

# CSV file containing the list of strings to search for
set strings_file = "/project/crism/users/phillms1/Murchie2017MDAP/tyrrhena_tile_ids.csv"

# Output file where the full path of the file containing the strings will be appended
set output_file = "/project/crism/users/phillms1/Murchie2017MDAP/tyrrhena_tile_id_paths.txt"

# Read the strings from the CSV file
set strings = `cat $strings_file`

# Search the current directory and all subdirectories for files containing the strings and "MRRSU" in their filename
foreach str ($strings)
  find . -type f -name "*$str*MRRSU*.IMG" -exec readlink -f {} \; >> $output_file
  #set files = `find . -type f -name "*$str*MRRSU*.IMG" -exec readlink -f {} \;`
  #echo $files >> $output_file
end

#foreach str ($strings)
#  set files = `find . -type f -name "*$str*MRRSU*.IMG"`
#  if ($#files >= 0) then
#    foreach file ($files)
#      set full_path = `readlink -f $file`
#      echo $full_path >> $output_file
#    end
#  endif
#end

You can see in the commented out lines I tried several different ways to do this.

I've tried just echo-ing the $str variable to make sure it contains what I thought, and it does. echo $str returns the values I expect. When I simply echo the find command, for example:

echo find . -type f -name "*$str*MRRSU*.IMG" -exec readlink -f {} \;

instead of returning the whole command it returns:

*MRRSU*.IMG" -exec readlink -f {} ;

and only the last iteration looks how it ought to look:

find . -type f -name "*0534*MRRSU*.IMG" -exec readlink -f {} ;

Does anyone know what's going on and how I can fix this?? Thank you!

Michael
  • 21
  • 1
  • Note that `for str in $strings` is an antipattern. See the various actions sections of [UsingFind](https://mywiki.wooledge.org/UsingFind) for advice on using `find` correctly. – Charles Duffy Feb 14 '23 at 00:35
  • That said, the _immediate_ problem is clearly is that you have DOS newlines -- the cursor going back to the left in the middle of a line is a complete giveaway, as that's what happens when a carriage return is written to the terminal. – Charles Duffy Feb 14 '23 at 00:36
  • If your editor is vim, run `:set fileformat=unix`, or use `dos2unix`, or see one of many other approaches described in the linked duplicate. You may also find it useful to run your script with xtrace logging (`bash -x yourscript`) and read the trace logs -- inside a `$''` string, `\r` is how a carriage return literal is printed, so wherever you see `\r`s you know there are carriage returns in your data. – Charles Duffy Feb 14 '23 at 00:38
  • Anyhow -- one _other_ aspect of DOS text files is that they use CRLFs to _separate_ lines, whereas UNIX uses a LF to _terminate_ a line, so in a DOS file you only have a CR on lines other than the very last one, hence it being only the last line that works the way you expect. (Whereas in the UNIX world, a line that doesn't have a LF after it isn't a valid line of text at all, and tools like `read` return false when the terminator it's told to expect -- a LF by default -- doesn't exist) – Charles Duffy Feb 14 '23 at 00:39
  • BTW, re: a separate / less-immediate problem, see [DontReadLinesWithFor](https://mywiki.wooledge.org/DontReadLinesWithFor). – Charles Duffy Feb 14 '23 at 00:40
  • Thank you Charles - running :set fileformat=unix on my .txt file that contained the string patters worked! I don't think I'm reading any lines with for in this example, although I am writing lines with for... I guess the same advice applies (use a while loop instead?) – Michael Feb 14 '23 at 15:36
  • The DontReadLinesWithFor was re: `for str in $strings` -- it relies on `$strings` being word-split and glob-expanded, and those behaviors aren't always what one intends when writing that code. – Charles Duffy Feb 14 '23 at 15:52

0 Answers0