-1

I have 2 text files, file1 and file2. I need to output those lines in file1 that do not exist in file2.

The lines in both files are not in the same order and there are additional lines in file2, which are irrelevant.

For example:

$ grep testme file1
chmod -f 644 /root/testme
chown -h root:root /root/testme

$ grep testme file2
chmod -f 777 /root/testme
chown -h dude:dude /root/testme

The above grep is an example of 2 lines in file1 that are different, or as such do not exist in file2. Both files are about 5 MB.

Using the following procedure in Linux Bash (RHEL 7)

while read -r line; do
 [ "$(\grep "^$line\$" file2 )" ] || echo $line
done < file1

displays...

chmod -f 644 /root/testme
chown -h root:root /root/testme
grep: Unmatched [ or [^`

... and won't finish. It also takes a very long time.

I tried another approach, which is very fast, but only prints 1 of the 2 lines:

$ fgrep -v -f file2 file1
chmod -f 644 /root/testme
$

Why is it only showing 1 line?

The 2 lines are somewhere in the middle of the file, terminated by a CR.

Any ideas please? Thanks!

agc
  • 7,973
  • 2
  • 29
  • 50
Dude
  • 113
  • 1
  • 10
  • 2
    [Get lines of file1 which are not in file2](http://stackoverflow.com/q/19201029/3776858) – Cyrus Apr 17 '17 at 09:06
  • please refer the following thread for the above query: http://stackoverflow.com/questions/14473090/find-lines-from-a-file-which-are-not-present-in-another-file. – Krishan Chawla Apr 17 '17 at 09:07
  • Since the input files need to be sorted, I was wondering if the absolut position of lines matter. Also the output will be sorted, which I would like to avoid. – Dude Apr 17 '17 at 09:35
  • You should incorporate this last comment to the question.. check my update – Juan Diego Godoy Robles Apr 17 '17 at 09:42
  • Why? The problem of sorting applies to comm, but not grep or fgrep. It's comm that requires sorting and which introduces the additional concern. It's not a requirement for me not to have unsorted output, but a preference, considering how comm sorts the output. Anyway, as far as I'm concerned, the problem seems to be solved by simply using -x as outlined. Thanks for helping! – Dude Apr 17 '17 at 19:28

2 Answers2

1

Many thanks for the help and suggestion to use comm, but meanwhile I found the answer using grep. I rather prefer grep, since it's much faster than comm and also does not require the input to be sorted.

$ fgrep -v -x -f file2 file1
chmod -f 644 /root/testme
chown -h root:root /root/testme

The solution is simply to add -x:

-x Select only those matches that exactly match the whole line

agc
  • 7,973
  • 2
  • 29
  • 50
Dude
  • 113
  • 1
  • 10
0

combine can also do this:

combine <(seq 5 10) not <(seq 1 2 20)
6
8
10
agc
  • 7,973
  • 2
  • 29
  • 50