Search using reference file and print matching lines

Question

have a folder structure as shown below ./all_files

-rwxrwxrwx  reference_file.txt
drwxrwxrwx  file1.txt
drwxrwxrwx  file2.txt
drwxrwxrwx  file3.txt

reference_file.txt has filenames as shown below

$cat reference_file.txt
file1.txt
file2.txt

data in file1.txt and file2.txt are as shown below:

$cat file1.txt
step_1
step_2
step_3

Now, I have to take particular step say step2 from each file

Note1: file name must present in reference_file.txt

Note2: step2 is not line no:2 always.

Note3: search should perform recursively.

I have used below script:

#!/bin/sh

for i in cat reference_file.txt;

do
   find . -type f -name $i | grep -v 'FS*' | xargs  grep -F 'step_2'
done<reference_file.txt

after using above code i got no output.

# bash -x script.sh
+ for i in cat reference_file.txt
+ find . -type f -name **cat**
+ xargs grep -F 'step_2'
+ for i in cat **reference_file.txt**
+ find . -type f -name reference_file.txt
+ xargs grep -F 'step_2'

Added New requirement:

target=step_XX_2 where XX can be anything and should be skipped for search.. so that desire ouput will be.. step_ab_2 step_cd_2 step_ef_2

Answer 1

I think this is what you are trying to achieve. Please let me know:

EDIT: my previous version did not search recursively. Further edits: Note that using process substitution for find means that this script MUST be run under bash and not sh.

Further edit for change in specification: note the change to target and the -E option to grep instead of -F.

#!/bin/bash

target='step_.*?_?2'

while read -r name
do
   # EDIT: exclude certain directories
   if [[ $name == "old1" || $name == "old2" ]]
   then
        # do the next iteration of the loop
        continue    
   fi

   while read -r fname
   do
       if [[ $fname != FS* ]]
       then
           # Display the filename (grep -H is not in POSIX)
           if out=$(grep -E "$target" "$fname")
           then
               echo "$fname: $out"
           fi
       fi
   done < <(find . -type f -name "$name")

done < reference_file.txt

Note that your trace (bash -x) uses bash but your #! line uses sh. They are different - you should be consistent with the shell you are using.

So, I have dropped the xargs, that reads strings standard input and executes a program using the strings as argument. Since we already have the argument strings for grep we don't need it.

Your grep -v 'FS*' probably doesn't do what you expect. The regular expression FS* means "F followed by zero or more S's". Not the same as a shell pattern matching (globbing). In my solution I have used FS* because I am using the shell, not grep.

Answer 2

I believe this question is duplicate of this

What you need is

#!/bin/sh

for i in `cat reference_file.txt`
    do  find . -type f -name $i | grep -v 'FS*' | xargs  grep -F 'step_2'  
done 

See the backticks and Do Not read the file reference_file.txt twice.