How to add the data of a column in csv file into array

Question

Say I have a csv file:

Col1,Col2,Col3,Col4
1,2,3,4
1,2,3,4
1,2,3,4
1,2,3,4

I want to add all values in a column into an array then manipulate it, then on to the next column

So:

# loop through data
Col1 = [1,1,1,1]
# do something
Col2 = [2,2,2,2]
# do something
Col3 = [3,3,3,3]
# do something
Col4 = [4,4,4,4]

The problem with using

data = csv.reader(input_file)
lst = []

for row in data:
    lst.append(row[0])
    # do something with lst

Is that I can only do that for the first column.

Please do not edit away your question's content. Either this question is useful as a landing pad for future searches, or you should delete it completely via the **delete** link at the bottom left. — Quentin, Apr 18 '16 at 08:11

score 0 · Answer 1 · edited May 23 '17 at 10:28

Check out this post by Ben Southgate: Extract csv file specific columns to list in Python

import csv

# open the file in universal line ending mode 
with open('test.csv', 'rU') as infile:
  # read the file as a dictionary for each row ({header : value})
  reader = csv.DictReader(infile)
  data = {}
  for row in reader:
    for header, value in row.items():
      try:
        data[header].append(value)
      except KeyError:
        data[header] = [value]

This code by him creates a dict with your lists. You can then access them by:

Col1 = data['Col1']

Would have put the link in a comment, but I don't have enough rep yet to comment.

score 0 · Answer 2 · answered Mar 27 '16 at 12:53

I would use numpy to read the entire csv at once and then you can just work with an array as follows:

import numpy as np
my_data = np.genfromtxt('test.csv', delimiter=',')
for column in my_data.T:
  print(column)

Which gives:

[ 1.  1.  1.  1.]
[ 2.  2.  2.  2.]
[ 3.  3.  3.  3.]
[ 4.  4.  4.  4.]

for a csv file like this:

1,2,3,4
1,2,3,4
1,2,3,4
1,2,3,4

score 0 · Answer 3 · answered Mar 27 '16 at 13:10

It seems you can read the file into a list of lists. If so, look at the zip function. It takes lists as arguments, and combines the first elements into a new list, second into a new list, etc.

>>> data = [[1,2,3,4],[1,2,3,4],[1,2,3,4]]
>>> transposed = zip(*data)
>>> transposed
[(1, 1, 1), (2, 2, 2), (3, 3, 3), (4, 4, 4)]
>>>

As was pointed out, numpy can do this (and more!) but it is an additional package not included with python.

score 0 · Answer 4 · answered Mar 27 '16 at 13:20

This reads the content into a dictionary:

import csv
import pprint

with open('cols.csv') as input_file:
    reader = csv.reader(input_file)
    col_names = next(reader)
    data = {name: [] for name in col_names}
    for line in reader:
        for pos, name in enumerate(col_names):
            data[name].append(int(line[pos]))

pprint.pprint(data)

Output:

{'Col1': [1, 1, 1, 1],
 'Col2': [2, 2, 2, 2],
 'Col3': [3, 3, 3, 3],
 'Col4': [4, 4, 4, 4]}

How to add the data of a column in csv file into array

4 Answers4