59

I realize that the csv library in Python always generates DOS end-of-line characters. Even if I use the 'wb' mode, even if I use Linux.

import csv

f = open('output.txt', 'wb');
writer = csv.writer(f)
writer.writerow([2,3,4]);
f.close()

The above code always uses '\r\n' as the end of line separator. How can I make it use use '\n' only?

Ross Ridge
  • 38,414
  • 7
  • 81
  • 112
CuriousMind
  • 15,168
  • 20
  • 82
  • 120
  • 10
    The CSV MIME spec mandates CRLF as the line delimiter: https://tools.ietf.org/html/rfc4180 – Richard Simões Apr 20 '16 at 19:52
  • I believe the csv *reader* always does and can't be changed, but you're only asking about the *writer*, whose default can be changed. – smci Jun 15 '16 at 20:56

2 Answers2

94

You can give your writer instance a custom lineterminator argument in the constructor:

writer = csv.writer(f, lineterminator="\n")
Niklas B.
  • 92,950
  • 18
  • 194
  • 224
  • can lineterminator be given as combination of two characters ? – user1479571 Sep 14 '16 at 23:23
  • @user1479571 If you follow the link to the documentation for `lineterminator`, it says that it is a string, and the default value is `'\r\n'`. So yes. – LarsH Nov 22 '19 at 13:53
34

As Niklas answered, the lineterminator argument lets you choose your line endings. Rather than hard coding it to '\n', make it platform independent by using your platform's line separator: os.linesep. Also, make sure to specify newline='' for Python 3 (see this comment).

import csv
import os

with open('output.csv', 'w', newline='') as f:
    writer = csv.writer(f, lineterminator=os.linesep)
    writer.writerow([2, 3, 4])

Old solution (Python 2 only)

In Python 2, use 'wb' mode (see the docs).

import csv
import os

with open('output.csv', 'wb') as f:
    writer = csv.writer(f, lineterminator=os.linesep)
    writer.writerow([2, 3, 4])

For others who find this post, don't miss the 'wb' if you're still using Python 2. (In Python 3, this problem is handled by Python). You won't notice a problem if you're missing it on some platforms like GNU/Linux, but it is important to open the file in binary mode on platforms where that matters, like Windows. Otherwise, the csv file can end up with line endings like \r\r\n. If you use the 'wb' and os.linesep, your line endings should be correct on all platforms.

vauhochzett
  • 2,732
  • 2
  • 17
  • 40
Don Kirkby
  • 53,582
  • 27
  • 205
  • 286
  • so even if you make the writer "*platform independent*", your script is still platform dependent because you need to open it in binary, but only where necessary ? If I try `wb` on linux I get an error for `writer.writeheader()` – Ciprian Tomoiagă Dec 21 '16 at 20:49
  • 1
    I just tried it on Ubuntu 16.04 using both `writer` and `DictWriter` with `wb`, @CiprianTomoiaga, and it works fine for me. I suggest you post a new question with a code sample and the error message, then add a comment here with a link to your question. – Don Kirkby Dec 21 '16 at 22:15
  • 5
    @CiprianTomoiaga This doesn't work with Python 3. The CSV module assumes that it can write normal strings and this isn't the case anymore if you open the file with `'wb'` (you would get a `TypError`). Thus, you have to open the file like this: `f = open('output.csv', 'w', newline='')` when you use Python 3. See also the [Python 3 CSV documentation](https://docs.python.org/3.5/library/csv.html#csv.writer). – maxschlepzig Jun 25 '17 at 22:02
  • 3
    Consider too that the main purpose of CSV in general is as an interchange format that works between dissimilar systems. So while using `lineterminator=os.linesep` will produce CSV that's suited to the system on which this script runs, that also means the CSV produced is different depending on where the script runs; neither of which is necessarily right for the system on which the CSV is consumed. So "platform independent" for the consumer might mean sticking to a single standard rather than varying with the OS of the producer. – LarsH Nov 22 '19 at 14:07
  • While I agree that using the OS separator is normally the best option, the one place I would use "\n" is when running Python in Git Bash on Windows, or you get mixed line terminators. – Huw Walters Dec 19 '22 at 11:56