Grep from variable stored in a csv file

Question

I'm try to grep for everything within "test.csv" from the contents on book.1

while IFS= read -r result
do
grep -r $result test.csv >> output.csv

echo $result

done<book1.csv

My book 1 has 1 row of data is column A.

I am getting the results of book1.csv shown on the screen so it is parsing correctly, but I'm getting nothing in the output.csv

I have manually checked the grep & there is contents of book1.csv within the test.csv

What have I done wrong?

EDIT

SAMPLE Input book1.csv

HOSTNAME
TEST1
TEST2

Sample input test.csv

blank,TEST1.abc.123,blank,date

I want to get every line from book1.csv that is in test.csv

Cat -a book1 result

TEST1^M$

when i do this "grep -r TEST1 test.csv >> output.csv" without a script i get the expected data so its something to do with my $result variable — Bunion, Sep 15 '17 at 09:00
`grep -Ff book1.csv test.csv` would usually work but you have to provide sample data from `test.csv` as well. Also check output of `cat -A book1.csv` and `cat -A test.csv` to make sure you don't have DOS carriage returns. — anubhava, Sep 15 '17 at 09:00
sample imput added, when i do the above grep -Ff i get nothing. However if i do grep TEST1 test.csv i get results so im not sure what is wrong — Bunion, Sep 15 '17 at 09:14

score 2 · Accepted Answer · answered Sep 15 '17 at 09:21

As suspected initially your .csv files have DOS line ending with \r characters, which is causing grep -f to not match since Unix files just end with \n.

You can use this command with tr that removes \r from both .csv files before running grep:

grep -Ff <(tr -d '\r' < book1.csv) <(tr -d '\r' < test.csv)

blank,TEST1.abc.123,blank,date

tr -d '\r' < book1.csv will delete \r or carriage return character from given files.

Charles · Answer 2 · 2017-09-15T09:29:38.780

using fgrep should work. With the data that you provided, it gives:

fgrep -f book1.csv test.csv

Output:

blank,TEST1.abc.123,blank,date

fgrep is strictly equivalent to grep -F

from grep man page :

Matcher Selection
   -F, --fixed-strings
          Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched.  (-F is specified by POSIX.)


Matching Control
   -f FILE, --file=FILE
          Obtain  patterns  from FILE, one per line.  The empty file contains zero patterns, and therefore matches nothing.  (-f is specified
          by POSIX.)

As pointed out by anubhava , you need to remove the DOS character. To do that, just use the dos2unix command:

fgrep -f <(dos2unix book1.csv) <(dos2unix test.csv)

Grep from variable stored in a csv file

2 Answers2