149

I need a conversion utility/script that will convert a .sql dump file generated on Mac to one readable on Windows. This is a continuation of a problem I had here. The issue seems to be with newline formatting in text files, but I can't find a tool to make the conversion...

Community
  • 1
  • 1
Yarin
  • 173,523
  • 149
  • 402
  • 512
  • 4
    A generic tool I made after not finding any satisfactory solution with industrial strength https://github.com/mdolidon/endlines – Mathias Dolidon May 09 '15 at 12:21

12 Answers12

147

Windows uses carriage return + line feed for newline:

\r\n

Unix only uses Line feed for newline:

\n

In conclusion, simply replace every occurence of \n by \r\n.
Both unix2dos and dos2unix are not by default available on Mac OSX.
Fortunately, you can simply use Perl or sed to do the job:

sed -e 's/$/\r/' inputfile > outputfile                # UNIX to DOS  (adding CRs)
sed -e 's/\r$//' inputfile > outputfile                # DOS  to UNIX (removing CRs)
perl -pe 's/\r\n|\n|\r/\r\n/g' inputfile > outputfile  # Convert to DOS
perl -pe 's/\r\n|\n|\r/\n/g'   inputfile > outputfile  # Convert to UNIX
perl -pe 's/\r\n|\n|\r/\r/g'   inputfile > outputfile  # Convert to old Mac

Code snippet from:
http://en.wikipedia.org/wiki/Newline#Conversion_utilities

Anne
  • 26,765
  • 9
  • 65
  • 71
  • 37
    The `sed` command for UNIX to DOS does not work for me on OS X Lion - it just inserts the text "r" at the end of each line. The `perl` command works though. – Ergwun May 18 '12 at 06:59
  • 7
    OSX uses older version of sed. I use Homebrew for OSX, and installed gnu-sed. You use with the "gsed" command instead of "sed". That works. – John Nov 11 '13 at 23:01
  • The perl versions don't work for me on windows, because the -p switch doesn't use `binmode`, and `readline` will convert the `"\r\n"` to `"\n"`. – gatinueta Mar 28 '14 at 08:48
  • As an aside, when I tried using this to fix a `hosts` file, this failed for me because perl and sed both couldn't write the file (despite using `sudo`). I had to write the file elsewhere and then move it into the `/etc` directory. – Kat Jul 24 '15 at 17:33
  • 3
    Use Homebrew to get the dos2unix and unix2dos packages instead. – Pratyush Aug 20 '15 at 14:47
  • 11
    OS X Yosemite still has the same problem with `sed`, but you can work around it without installing Homebrew, gnu-sed or unix2dos: Use `sed -e 's/$/^M/' inputfile > outputfile`, where `^M` is a control character produced on the command line via `Ctrl+V Ctrl+M`. – LarsH Oct 05 '15 at 18:26
  • 3
    Another workaround for Mac OS (tested on 10.13.6 High Sierra): Place a `$` before the single quote containing the sed command: `sed $'s/\r$//'` Explanation: bash decodes backslash-escapes in `$'...'` strings. See https://www.gnu.org/software/bash/manual/html_node/ANSI_002dC-Quoting.html for details. – jcsahnwaldt Reinstate Monica Jan 22 '19 at 01:41
136

This is an improved version of Anne's answer -- if you use perl, you can do the edit on the file 'in-place' rather than generating a new file:

perl -pi -e 's/\r\n|\n|\r/\r\n/g' file-to-convert  # Convert to DOS
perl -pi -e 's/\r\n|\n|\r/\n/g'   file-to-convert  # Convert to UNIX
Jeremy Friesner
  • 70,199
  • 15
  • 131
  • 234
JosephH
  • 37,173
  • 19
  • 130
  • 154
  • 5
    The awesome thing about these scripts is that they show, with the regular expressions, EXACTLY what the end-of-line conversion needs to be to convert to either format, starting from anything. – pbr Jan 20 '13 at 21:20
  • be careful with this on certain Cygwin/git bash installations on Windows systems. This may give you `Can't do inplace edit on file: Permission denied.`, and **delete** the file. Look into other utilities instead. – Dennis Feb 12 '14 at 17:59
  • Huge thanks for showing "Convert to Unix". I was after that way and your double answer helped me, and got my upvote. – null Jan 12 '15 at 01:05
129

You can install unix2dos with Homebrew

brew install unix2dos

Then you can do this:

unix2dos file-to-convert

You can also convert dos files to unix:

dos2unix file-to-convert
Steven Hirlston
  • 1,869
  • 1
  • 15
  • 19
  • 10
    For anyone that comes across this now, the Homebrew formula is now called `dos2unix`. You'll want to `brew install dos2unix`. – Geoff Apr 26 '16 at 19:06
  • 14
    Actually, either `brew install unix2dos` or `brew install dos2unix` work fine. They install the same package. Use whichever name speaks to you :) – Steven Hirlston May 19 '16 at 16:44
  • 2
    Or with [Macports](https://www.macports.org/): `port install dos2unix`. – Fang Sep 02 '16 at 11:46
17

Just do tr delete:

tr -d "\r" <infile.txt >outfile.txt
Szymon
  • 42,577
  • 16
  • 96
  • 114
parahren
  • 179
  • 1
  • 5
  • 2
    Tried perl and sed, didn't work (I could have figured it out, wasn't worth a try). This worked great. – RandomInsano Jun 04 '14 at 18:43
  • This was the first solution I found to BBEdit's line numbers not matching the count of lines as I read them using Python (and not matching `wc -l`). – Daryl Spitzer Jul 18 '14 at 04:36
  • 1
    this deletes all of the line breaks I actually still need to have line breaks but with \n – UserYmY Feb 10 '15 at 13:36
  • "http://hints.macworld.com/article.php?story=20031018164326986" also has a good write-up on how to use the `tr` command to perform various conversions. Use `hexdump` or similar to find out exactly what sort of end-of-line convention is now used in the file. – Mike Robinson May 04 '15 at 20:41
17

You probably want unix2dos:

$ man unix2dos

NAME
       dos2unix - DOS/MAC to UNIX and vice versa text file format converter

SYNOPSIS
           dos2unix [options] [-c CONVMODE] [-o FILE ...] [-n INFILE OUTFILE ...]
           unix2dos [options] [-c CONVMODE] [-o FILE ...] [-n INFILE OUTFILE ...]

DESCRIPTION
       The Dos2unix package includes utilities "dos2unix" and "unix2dos" to convert plain text files in DOS or MAC format to UNIX format and vice versa.  Binary files and non-
       regular files, such as soft links, are automatically skipped, unless conversion is forced.

       Dos2unix has a few conversion modes similar to dos2unix under SunOS/Solaris.

       In DOS/Windows text files line endings exist out of a combination of two characters: a Carriage Return (CR) followed by a Line Feed (LF).  In Unix text files line
       endings exists out of a single Newline character which is equal to a DOS Line Feed (LF) character.  In Mac text files, prior to Mac OS X, line endings exist out of a
       single Carriage Return character. Mac OS X is Unix based and has the same line endings as Unix.

You can either run unix2dos on your DOS/Windows machine using cygwin or on your Mac using MacPorts.

Pang
  • 9,564
  • 146
  • 81
  • 122
Paul R
  • 208,748
  • 37
  • 389
  • 560
  • unix2dos/dos2unix do not exist on my mac and I haven't found any place to install them- Do you know of any? – Yarin Jun 16 '11 at 17:00
  • @mgadda: +1 - yes, I switched to homebrew from MacPorts a while back now and haven't looked back. – Paul R Jun 28 '13 at 07:07
9
  1. Install dos2unix with homebrew
  2. Run find ./ -type f -exec dos2unix {} \; to recursively convert all line-endings within current folder
Colin
  • 1,987
  • 3
  • 17
  • 21
AAverin
  • 3,014
  • 3
  • 27
  • 32
5

vim also can convert files from UNIX to DOS format. For example:

vim hello.txt <<EOF
:set fileformat=dos
:wq
EOF

Conversely, if you need to go from DOS to UNIX:

vim hello.txt <<EOF
:set fileformat=unix
:wq
EOF
Stephen Quan
  • 21,481
  • 4
  • 88
  • 75
2

Here's a really simple approach, worked well for me, courtesy Davy Schmeits's Weblog:

cat foo | col -b > foo2

Where foo is the file that has the Control+M characters at the end of the line, and foo2 the new file you are creating.

patdevelop
  • 41
  • 1
2

The following is a complete script based on the above answers along with sanity checking and works on Mac OS X and should work on other Linux / Unix systems as well (although this has not been tested).

#!/bin/bash

# http://stackoverflow.com/questions/6373888/converting-newline-formatting-from-mac-to-windows

# =============================================================================
# =
# = FIXTEXT.SH by ECJB
# =
# = USAGE:  SCRIPT [ MODE ] FILENAME
# =
# = MODE is one of unix2dos, dos2unix, tounix, todos, tomac
# = FILENAME is modified in-place
# = If SCRIPT is one of the modes (with or without .sh extension), then MODE
# =   can be omitted - it is inferred from the script name.
# = The script does use the file command to test if it is a text file or not,
# =   but this is not a guarantee.
# =
# =============================================================================

clear
script="$0"
modes="unix2dos dos2unix todos tounix tomac"

usage() {
    echo "USAGE:  $script [ mode ] filename"
    echo
    echo "MODE is one of:"
    echo $modes
    echo "NOTE:  The tomac mode is intended for old Mac OS versions and should not be"
    echo "used without good reason."
    echo
    echo "The file is modified in-place so there is no output filename."
    echo "USE AT YOUR OWN RISK."
    echo
    echo "The script does try to check if it's a binary or text file for sanity, but"
    echo "this is not guaranteed."
    echo
    echo "Symbolic links to this script may use the above names and be recognized as"
    echo "mode operators."
    echo
    echo "Press RETURN to exit."
    read answer
    exit
}

# -- Look for the mode as the scriptname
mode="`basename "$0" .sh`"
fname="$1"

# -- If 2 arguments use as mode and filename
if [ ! -z "$2" ] ; then mode="$1"; fname="$2"; fi

# -- Check there are 1 or 2 arguments or print usage.
if [ ! -z "$3" -o -z "$1" ] ; then usage; fi

# -- Check if the mode found is valid.
validmode=no
for checkmode in $modes; do if [ $mode = $checkmode ] ; then validmode=yes; fi; done
# -- If not a valid mode, abort.
if [ $validmode = no ] ; then echo Invalid mode $mode...aborting.; echo; usage; fi

# -- If the file doesn't exist, abort.
if [ ! -e "$fname" ] ; then echo Input file $fname does not exist...aborting.; echo; usage; fi

# -- If the OS thinks it's a binary file, abort, displaying file information.
if [ -z "`file "$fname" | grep text`" ] ; then echo Input file $fname may be a binary file...aborting.; echo; file "$fname"; echo; usage; fi

# -- Do the in-place conversion.
case "$mode" in
#   unix2dos ) # sed does not behave on Mac - replace w/ "todos" and "tounix"
#       # Plus, these variants are more universal and assume less.
#       sed -e 's/$/\r/' -i '' "$fname"             # UNIX to DOS  (adding CRs)
#       ;;
#   dos2unix )
#       sed -e 's/\r$//' -i '' "$fname"             # DOS  to UNIX (removing CRs)
#           ;;
    "unix2dos" | "todos" )
        perl -pi -e 's/\r\n|\n|\r/\r\n/g' "$fname"  # Convert to DOS
        ;;
    "dos2unix" | "tounix" )
        perl -pi -e 's/\r\n|\n|\r/\n/g'   "$fname"  # Convert to UNIX
        ;;
    "tomac" )
        perl -pi -e 's/\r\n|\n|\r/\r/g'   "$fname"  # Convert to old Mac
        ;;
    * ) # -- Not strictly needed since mode is checked first.
        echo Invalid mode $mode...aborting.; echo; usage
        ;;
esac

# -- Display result.
if [ "$?" = "0" ] ; then echo "File $fname updated with mode $mode."; else echo "Conversion failed return code $?."; echo; usage; fi
ECJB
  • 59
  • 3
0

On Yosemite OSX, use this command:

sed -e 's/^M$//' -i '' filename

where the ^M sequence is achieved by pressing Ctrl+V then Enter.

ByteHamster
  • 4,884
  • 9
  • 38
  • 53
Olga
  • 9
  • 1
  • Also note that `sed` *does* understand backslash-escapes such as `\r` and ``\n` and therefore can also use these in the substitution. You don't actually have to input a literal control-M to refer to that character (or any other). The principle of using `sed` (and `-i`) to do *any* sort of conversion of this kind is a very good one, because, unlike `tr`, you are not limited to "one character at a time." – Mike Robinson May 04 '15 at 20:44
0

Expanding on the answers of Anne and JosephH, using perl in a short perl script, since i'm too lazy to type the perl-one-liner very time.
Create a file, named for example "unix2dos.pl" and put it in a directory in your path. Edit the file to contain the 2 lines:

#!/usr/bin/perl -wpi
s/\n|\r\n/\r\n/g;

Assuming that "which perl" returns "/usr/bin/perl" on your system. Make the file executable (chmod u+x unix2dos.pl).

Example:
$ echo "hello" > xxx
$ od -c xxx (checking that the file ends with a nl)
0000000 h e l l o \n

$ unix2dos.pl xxx
$ od -c xxx (checking that it ends now in cr lf)
0000000 h e l l o \r \n

avy
  • 1
0

In Xcode 9 in the left panel open/choose your file in project navigator. If file is not there, drug-and-drop it into the project navigator.

On right panel find Text Settings and change Line Endings to Windows (CRLF) .

XCode screendumpscreendump from XCode

matrix3003
  • 386
  • 3
  • 9