Looping and grep writes output for the last line only

Question

I am looping through the lines in a text file. And performing grep on each lines through directories. like below

while IFS="" read -r p || [ -n "$p" ]
do
 echo "This is the field: $p"
  grep -ilr $p * >> Result.txt
done < fields.txt

But the above writes the results for the last line in the file. And not for the other lines.

If i manually execute the command with the other lines, it works (which mean the match were found). Anything that i am missing here? Thanks

The fields.txt looks like this

annual_of_measure__c
attached_lobs__c
apple

The result.txt have the list of file names that has the string. And i am expecting the results like this for all the fields/lines in the fields.txt. — OK999, Oct 02 '18 at 15:09
Just an idea: is the fields.txt file from DOS with DOS lineendings and no newline in the last line? Can you check if the lineendings are unix lineendings? — Lars Fischer, Oct 02 '18 at 15:11
man, i think thats the answer... i remember encountering something like this before. i will check and confirm — OK999, Oct 02 '18 at 15:12
that was it @LarsFischer. Please update the answer and i will accept it — OK999, Oct 02 '18 at 15:18
If you fix fields.txt to use linefeeds instead of linefeed/carriage-return pairs as line endings, you won't need the `|| [ -n "$p" ]` hack anymore either. — chepner, Oct 02 '18 at 15:28

Lars Fischer · Accepted Answer · 2018-10-02T15:42:34.697

When the file fields.txt

has DOS/Windows lineending convention consisting of two character (Carriage-Return AND Linefeed) and
that file is processed by Unix-Tools expecting Unix lineendings consisting of only one character (Linefeed)

then the line read by the read command and stored in the variable $p is in the first line annual_of_measure__c\r (note the additional \r for the Carriage-Return). Then grep will not find a match.

From your description in the question and the confirmation in the comments, it seems that the last line in fields.txt has no lineending at all, so the variable $p is the ordinary string apple and grep can find a match on the last line of the file.

There are tools for converting lineendings, e.g. see this answer or even more options in this answer.

The above explanation is spot on. I am using a pc and the file have the DOS lineending convention. The same file is process in a bash shell (git bash on windows). i used the dos2unix executable to convert the file. — OK999, Oct 02 '18 at 15:41

Looping and grep writes output for the last line only

1 Answers1