87

I am trying to access a model.filefield in Django to parse a CSV file in Python using the csv module. It's working on Windows, but on Mac it gave me this:

Exception Type: Error

Exception Value: new-line character seen in unquoted field - do you need to open the file in universal-newline mode?

This is the code:

myfile = customerbulk.objects.all()[0].fileup

mydata = csv.reader(myfile)
    for email,mobile,name,civilid in mydata:
        print email,mobile,name,civilid
jdhao
  • 24,001
  • 18
  • 134
  • 273
mohd
  • 2,634
  • 2
  • 16
  • 18
  • what is this `customerbulk.objects.all()[0].fileup` thing. Is it a filename on a model? – SingleNegationElimination Jul 17 '11 at 21:42
  • fileup = models.FileField(verbose_name="CsvFile",upload_to='ExcelFiles') if i make a small query like customerbulk.objects.get(pk=1) – mohd Jul 17 '11 at 21:49
  • 1
    The exact reason of using `rU` (It's related to the open() function and is not csv specific.): `In addition to the standard fopen() values mode may be 'U' or 'rU'. Python is usually built with universal newlines support; supplying 'U' opens the file as a text file, but lines may be terminated by any of the following: the Unix end-of-line convention '\n', the Macintosh convention '\r', or the Windows convention '\r\n'.` http://docs.python.org/2/library/functions.html#open – zengr Jun 04 '13 at 02:53
  • In Python >= 3, use `newline=''` instead of `mode='U'`. – tricasse Jan 19 '18 at 14:37

1 Answers1

151

I finally found the solution:

mypath = customerbulk.objects.get(pk=1).fileup.path
o = open(mypath,'rU')
mydata = csv.reader(o)
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
mohd
  • 2,634
  • 2
  • 16
  • 18
  • 2
    I believe you can simplify this. It seems odd to seek to the start after just opening the file. The following helped me overcome the same issue from an Excel spreadsheet export to CSV using the defaults: data = csv.reader(open(FILENAME, 'rU'), quotechar='"', delimiter = ',') – timbo May 15 '13 at 23:48
  • 4
    Yes, there's no need to seek to beginning of file just after opening. The >>>o = open(mypath, 'rU') bit, with the 'rU' flag is the important magic that worked in my case. – rd108 May 31 '13 at 20:18
  • 13
    [PEP278](http://www.python.org/dev/peps/pep-0278/) explained what `rU` stands for:`In a Python with universal newline support open() the mode parameter can also be "U", meaning "open for input as a text file with universal newline interpretation". Mode "rU" is also allowed, for symmetry with "rb".` – Xiao Dec 03 '13 at 07:44
  • @Xiao +1 for linking to the original PEP which explains the rationale. – Naymesh Mistry Aug 15 '17 at 22:15
  • In Python >= 3, use `newline` instead, the default is `newline=None` which is like `newline=''` except it also translates the newlines to `\n`. I'm unsure which of these is equivalent to `mode='U'` – Tim Diels Jan 31 '19 at 16:42