How to write Unix end of line characters in Windows?

Question

How can I write to files using Python (on Windows) and use the Unix end of line character?

e.g. When doing:

f = open('file.txt', 'w')
f.write('hello\n')
f.close()

Python automatically replaces \n with \r\n.

Answer 1

The modern way: use newline=''

Use the newline= keyword parameter to io.open() to use Unix-style LF end-of-line terminators:

import io
f = io.open('file.txt', 'w', newline='\n')

This works in Python 2.6+. In Python 3 you could also use the builtin open() function's newline= parameter instead of io.open().

The old way: binary mode

The old way to prevent newline conversion, which does not work in Python 3, is to open the file in binary mode to prevent the translation of end-of-line characters:

f = open('file.txt', 'wb')    # note the 'b' meaning binary

but in Python 3, binary mode will read bytes and not characters so it won't do what you want. You'll probably get exceptions when you try to do string I/O on the stream. (e.g. "TypeError: 'str' does not support the buffer interface").

Answer 2

For Python 2 & 3

See: The modern way: use newline='' answer on this very page.

For Python 2 only (original answer)

Open the file as binary to prevent the translation of end-of-line characters:

f = open('file.txt', 'wb')

Quoting the Python manual:

On Windows, 'b' appended to the mode opens the file in binary mode, so there are also modes like 'rb', 'wb', and 'r+b'. Python on Windows makes a distinction between text and binary files; the end-of-line characters in text files are automatically altered slightly when data is read or written. This behind-the-scenes modification to file data is fine for ASCII text files, but it’ll corrupt binary data like that in JPEG or EXE files. Be very careful to use binary mode when reading and writing such files. On Unix, it doesn’t hurt to append a 'b' to the mode, so you can use it platform-independently for all binary files.

Answer 3

You'll need to use the binary pseudo-mode when opening the file.

f = open('file.txt', 'wb')

Answer 4

def dos2unix(inp_file, out_file=None):
    if out_file:
        out_file_tmp = out_file
    else:
        out_file_tmp = inp_file + '_tmp'
        if os.path.isfile(out_file_tmp):
            os.remove(out_file_tmp)
    with open(out_file_tmp, "w", newline='\n') as fout: 
        with open(inp_file, "r") as fin:
            lines = fin.readlines()
            lines = map(lambda line: line.strip() + '\n', lines)
            fout.writelines(lines)
    if not out_file:
        shutil.move(out_file_tmp, inp_file)
        print(f'dos2unix() {inp_file} is overwritten with converted data !')
    else:
        print(f'dos2unix() {out_file} is created with converted data !')