Parsing a csv file in python django

Question

I am trying to read data from a csv file that has been uploaded. First, I am getting each row then trying to read data from each row by splitting it with comma, which is good for ideal case but if contains "," like address field, it will parse data in wrong format.

I would like to have a more reliable solution for val = v.split(',')

My code is

 upload_file = request.FILES['upload_file']
    data = [row for row in csv.reader(upload_file.read().splitlines())]

    for v in data:
       # v is every row
       val = v.split(',') #spliting value of every row to get each record of every row

Check out the [csv module](https://docs.python.org/3.4/library/csv.html). — Cody Piersall, Feb 07 '15 at 07:00
The `csv.reader` is **already** doing the splitting-by-comma and more -- why exactly are you finding any added value at all in that `v.split?! — Alex Martelli, Feb 07 '15 at 07:00

alacy · Accepted Answer · 2015-02-07T18:25:46.673

7

If you read in the file using a simple read statement like:

data = upload_file.read()

# you can use the re library --> import re
rows = re.split('\n', data) # splits along new line
for index, row in enumerate(rows):
    cells = row.split(',')
    # do whatever you want with the cells in each row
    # this also keeps an index if you want the cell's row index

Or you can use the csv.reader module:

file_reader = csv.reader(upload_file, delimiter=',')
for row in file_reader:
    # do something with row data.
    print(row)
    # would print the rows like
    # I, like, to, ride, my, bicycle
    # I, like, to, ride, my, bike

edited Feb 07 '15 at 18:25

answered Feb 07 '15 at 13:48

alacy

4,972
8
30
47

Good point using the stream consumer from csv module. Cuts down on data manipulation and memory footprint. But shouldn't it just be csv.reader(upload_file, delimiter=',')? – spectras Feb 07 '15 at 15:58
@spectras ah good catch. I wrote this too quickly. I'll edit. – alacy Feb 07 '15 at 18:25

score 0 · Answer 2 · answered Feb 07 '15 at 07:04

0

If you're looking to split and get access to the words in each line then re.split would be a good option :

re.split('\W+', 'Words, words, words.')
['Words', 'words', 'words', '']

Sample code from : https://docs.python.org/2/library/re.html

answered Feb 07 '15 at 07:04

nitimalh

919
10
26

Does not answer the issue, as he pointed he must support fields that contain commas. That means being able to support quote-enclosed fields. – spectras Feb 07 '15 at 15:56

score 0 · Answer 3 · answered Feb 07 '15 at 14:28

CSV means comma separated values. If you need to encode a string in a CSV, you usually surround it with quotes. Otherwise you won't be able to parse correctly the file:

$ cat sample.csv 
"131, v17",foo
bar,qux


>>> import csv
>>> with open('sample.csv', 'rb') as f:
...   r = csv.reader(f)
...   for row in r:
...     print row
... 
['131, v17', 'foo']
['bar', 'qux']

And of course if you ommit the quotes, the first line is parsed into 3 fields.

score -1 · Answer 4 · edited May 23 '17 at 11:48

You could use pandas. Here's an example based on this question:

>>> import sys, pandas
>>> if sys.version_info[0] < 3:
    from StringIO import StringIO
else:
    from io import StringIO
## I assume your input is something like this:
>>> string = "a,b,c\n1,2,3\n4,5,6\n7,8,9\n"
>>> stringIO = StringIO(string)
>>> df = pandas.DataFrame.from_csv(stringIO, sep=',', index_col=False)
>>> print df
   a  b  c
0  1  2  3
1  4  5  6
2  7  8  9

>>> print df.columns
Index([u'a', u'b', u'c'], dtype='object')

## access elements
>>> print df['a'][3]
7

Documentation for DataFrame.from_csv

Parsing a csv file in python django

4 Answers4