Compare/Diff two files with different line terminators

Question

I have two text files I wish to make sure are the same, the problem is that file1 (SELECT_20150210.txt) is generated on a windows platform, and file2 (sel.txt) is generated on a mac, so the two files have different line terminating characters even though they look the same:

The first line:

Eriks-MacBook-Air:hftdump erik$ head -n 1 sel.txt
SystemState 0x04    25  03:03:48.800    O
Eriks-MacBook-Air:hftdump erik$ head -n 1 SELECT_20150210.txt
SystemState 0x04    25  03:03:48.800    O

cmp says they are different:

Eriks-MacBook-Air:hftdump erik$ cmp sel.txt SELECT_20150210.txt
sel.txt SELECT_20150210.txt differ: char 35, line 1

But it's only the terminating characters that differ:

Eriks-MacBook-Air:hftdump erik$ head -n 1 SELECT_20150210.txt | hexdump -C
00000000  53 79 73 74 65 6d 53 74  61 74 65 09 30 78 30 34  |SystemState.0x04|
00000010  09 32 35 09 30 33 3a 30  33 3a 34 38 2e 38 30 30  |.25.03:03:48.800|
00000020  09 4f 0d 0a                                       |.O..|
00000024
Eriks-MacBook-Air:hftdump erik$ head -n 1 sel.txt | hexdump -C
00000000  53 79 73 74 65 6d 53 74  61 74 65 09 30 78 30 34  |SystemState.0x04|
00000010  09 32 35 09 30 33 3a 30  33 3a 34 38 2e 38 30 30  |.25.03:03:48.800|
00000020  09 4f 0a                                          |.O.|
00000023

So is there a way to cmp or diff these two file and telling cmp to ignore the different line terminating character? Thank you

http://stackoverflow.com/questions/543779/line-end-agnostic-diff — afenster, Jul 04 '15 at 13:03

score 2 · Accepted Answer · edited May 23 '17 at 11:58

2

^{ASSUMPTION: you don't want to alter the line-endings of the original files}

To avoid creating temporary files, you could use process substitution:

diff my_unix_file <(dos2unix < my_dos_file)
diff my_unix_file <(sed 's/\r//' my_dos_file)
diff my_unix_file <(tr -d '\r' < my_dos_file)

^{UPDATE (Comments converted into answer): Some improvements done thanks to anishsane}

edited May 23 '17 at 11:58

Community

1
1

answered Jul 04 '15 at 13:06

Eugeniu Rosca

5,177
16
45

@anubhava: choose any of the dos2unix methods using `tr`, `awk`, `sed`, [etc](http://stackoverflow.com/questions/2613800/how-to-convert-dos-windows-newline-crlf-to-unix-newline-n-in-bash-script) – Eugeniu Rosca Jul 04 '15 at 13:45
Don't use `<(cat filename)`, when you just wanted `filename` :P `diff my_unix_file <(tr -d '\r' < my_dos_file)` – anishsane Jul 04 '15 at 14:14
@anishsane: thanks (will keep that in mind). answer updated. – Eugeniu Rosca Jul 04 '15 at 14:21

score 2 · Answer 2 · answered Jul 04 '15 at 13:10

2

On OSX you can use this diff:

diff osx-file.txt <(tr -d '\r' < win-file.txt)

tr -d '\r' < win-file.txt will strip r from win-file.txt.

answered Jul 04 '15 at 13:10

anubhava

761,203
64
569
643

Compare/Diff two files with different line terminators

2 Answers2