csv module broken for comma as delimiter

Question

I am using python's csv module and using quotes to contain the field that contains the separator.

The file a.txt looks like the following:

ab, ",cd"

And i get the following that indicates that it did not escape the comma in double quotes.

In [27]: with open('a.txt', newline='') as csvfile:
    ...:     reader = csv.reader(csvfile, delimiter=',')
    ...:     for row in reader:
    ...:         if row:
    ...:             print(row)
    ...:
['ab', ' "', 'cd"']

The space is throwing off the csv module's parsing. With `ab,",cd"` it works as expected. What's producing this bunk csv data? — wim, Sep 25 '19 at 20:33

zvone · Answer 1 · 2019-09-25T20:49:41.127

3

As @wim pointed out in a comment, this is caused by the space character after the comma.

If you configure the reader to skip the space after the delimiter, then it will work:

>>> print(list(csv.reader(io.StringIO('ab, ",cd"'), skipinitialspace=True)))
[['ab', ',cd']]

BTW, see the documentation for other CSV reader options.

edited Sep 25 '19 at 20:49

answered Sep 25 '19 at 20:48

zvone

18,045
3
49
77

1

What happens is that the csv module considers the quote _right after the separator_. If there's something before that (space or something else), the quote is considered like being part of the data. your answer allows to fix this – Jean-François Fabre Sep 25 '19 at 20:49

score 1 · Accepted Answer · answered Sep 25 '19 at 20:51

It's not broken, it's the expected behaviour.

What happens is that the csv module only considers the quote as a quote if it's right after the separator.

If there's something before that (space or something else), the quote is considered like being part of the data.

zvone answer explains how to workaround that (in the case of a space only)

csv module broken for comma as delimiter

2 Answers2