11

I have found that many of my files have DOS line endings. In VI they look like this: "^M". I don't want to modify files that don't have these DOS line endings. How do I do this using a bash script? Thanks!

EV

exvance
  • 1,339
  • 4
  • 13
  • 31

8 Answers8

19
grep -URl ^M . | xargs fromdos

grep gets you a list of all files under the current directory that have DOS line endings.

-U makes grep consider line endings instead of stripping them away by default

-R makes it recursive

-l makes it list only the filenames and not the matching lines

then you're piping that list into the converter command (which is fromdos on ubuntu, dos2unix where i come from).

NOTE: don't actually type ^M. instead, you'll need to press <Ctrl-V> then <Ctrl-M> to insert the ^M character and make grep understand what you're going for. or, you could type in $'\r' in place of ^M (but i think that may only work for bash...).

Nicolas Raoul
  • 58,567
  • 58
  • 222
  • 373
nullrevolution
  • 3,937
  • 1
  • 18
  • 20
5

One way using GNU coreutils:

< file.txt tr -d '\r'
Steve
  • 51,466
  • 13
  • 89
  • 103
4

On ubuntu, you use the fromdos utility

fromdos test.txt

The above example would take a MS-DOS or Microsoft Windows file or other file with different line separators and format the file with new line separators to be read in Linux and Unix.

Fredrik Pihl
  • 44,604
  • 7
  • 83
  • 130
1

Many options are there..you can try with any of these.. http://www.theunixschool.com/2011/03/different-ways-to-delete-m-character-in.html

Guru
  • 16,456
  • 2
  • 33
  • 46
1
cat origin_file.txt | sed "s/^M//" > dest_file.txt

You have to do the same thing mentioned above, ctl-V then ctl-M to get that character. This is preferable for me because it is portable across many platforms and keeps it simple within bash.

on ubuntu I also find this works:

cat origin_file.txt | sed "s/\r//" > dest_file.txt

Jim McNeely
  • 798
  • 5
  • 7
0

you can use the command:

   dos2ux file.in>file.out or:

in perl:

perl -pi -e 's/\r//g' your_file

alternatively you can do:

  • open in vi
  • go to command mode
  • type :%s/[ctrl-V][CTRL-M]//g
Vijay
  • 65,327
  • 90
  • 227
  • 319
0

Note if you're converting multi-byte files you need to take extra care, and should probably try to use the correct iconv or recode from-encoding specifications.

If it's a plain ASCII file, both of the below methods would work.

The flip program, in Debian the package is also called flip, can handle line-endings. From the manual:

When asked to convert a file to the same format that  it already 
has, flip  causes  no change to the file. Thus to convert all
files to **IX format you can type

flip -u *

and all files will end up right, regardless of whether they were 
in MS-DOS or in **IX format to begin with. This also works in the
opposite direction.

Or you could use GNU recode:

< /etc/passwd recode ..pc | tee a b > /dev/null
file a b

Output:

a: ASCII text, with CRLF line terminators
b: ASCII text, with CRLF line terminators

Convert to unix line-endings:

recode pc.. a b
file a b

Output:

a: ASCII text
b: ASCII text

recode abbreviates dos line-endings as pc, so the logic with pc.. is: convert from pc format to the default, which is latin1 with unix line-endings.

Thor
  • 45,082
  • 11
  • 119
  • 130
0

A modification to the Winning answer if you need to filter by file ending

grep -URl ^M . | grep .php | xargs dos2unix

I used dos2unix instead of fromdos but the effect should be the same.

Jarrod Juleff
  • 75
  • 8