15

I created a python script which works with a test CSV dataset of 10 records. When I scaled this up to the actual datasets (a few thousand rows), I am getting the following error:

_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

The code is as follows:

with open('./Origins.csv', 'r') as csvfile:
    reader = csv.DictReader(csvfile)
    origincoords = ['{Y},{X}'.format(**row) for row in reader]

The full error code is:

Traceback (most recent call last):
  File "./Driving.py", line 14, in <module>
    origincoords = ['{Y},{X}'.format(**row) for row in reader]
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 103, in next
    self.fieldnames
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/csv.py", line 90, in fieldnames
    self._fieldnames = self.reader.next()
_csv.Error: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

Perhaps there is a scale problem with the CSV reading method I am using?

martineau
  • 119,623
  • 25
  • 170
  • 301
LearningSlowly
  • 8,641
  • 19
  • 55
  • 78

1 Answers1

41

From PEP-0278:

In a Python with universal newline support open() the mode parameter can also be "U", meaning "open for input as a text file with universal newline interpretation". Mode "rU" is also allowed, for symmetry with "rb"

So try to change

with open('./Destinations.csv', 'r') as csvfile:

to

with open('./Destinations.csv', 'rb') as csvfile:

If the error persists, change to

with open('./Destinations.csv', 'rU') as csvfile:

Edited accorded to Martijn Pieters's comment.

Community
  • 1
  • 1
fredtantini
  • 15,966
  • 8
  • 49
  • 55