Parse CSV records into a list of Classes

Question

My first post on StackOverflow, though I've been a reader for a few years now!

I'm using Python to sanitize and analyse CSV data dumps of IV Curves.

My basic problem is the format that the datalogger gives me: every few minutes, at a single instance in time, it takes about 100 measurements of Voltage (v), Current (i), and Power (p), and dumps them into a CSV file. The next measurement gets appended to this. So, the structure of the data that we get is:

Date1;0.2;0.1;0.02
Date1;0.3;0.1;0.03
Date1;0.4;0.1;0.04
Date2;0.2;0.1;0.02
Date2;0.3;0.1;0.03
Date2;0.4;0.1;0.04
Date3; etc...

This data is stored in a file Data.csv

I've written a class called IVCurve:

class IVCurve:
    def __init__(self, datetime):
        self.datetime = datetime
    v = []
    i = []
    p = []

and I want to create a list of these class instances as:

count = -1
thelist = []
prev_time = 0

import csv

with open('Data.csv', 'rb') as IVdump:
    IVdata = csv.reader(IVdump, delimiter=';')
    for datetime, v, i, p in IVdata:
        # if we're onto a new date
        if prev_time != datetime:
            # note the new record
            prev_time=datetime
            #go to the next position in thelist[]
            count +=1
            #create a new curve
            thelist.append(IVCurve(datetime))
        # in any case, put the values of v, and i into this thelist[count]
        thelist[count].v.append(float(v))
        thelist[count].i.append(float(i))
        thelist[count].p.append(float(v)*float(i))

The problem I'm having is that all the values of v and i are placed in EVERY instance of thelist[], i.e., I'm getting a list of IVCurve instances, each with a different datetime, but each with the same set of v, i, and p (and that set represents the entire dataset for all dates combined).

I don't understand what I've done wrong here. Seems to me that every time count is incremented (each time we find a new date), thelist[count] should be a new, unique record.

Sample data (which I've used with this code) I've pasted here: http://pastebin.com/Dw4dd7tu

So the question is: how can I separate the values?

Thanks alot for your help!

Do not set class attributes to mutables, see [How do I avoid having Python class data shared among instances?](http://stackoverflow.com/q/1680528) `v = []`, etc. is going to lead to problems. — Martijn Pieters, Apr 04 '13 at 17:17
Martijn, thanks for your speedy reply! I read that post, and changed the class to put v, i, p in the init function, however now I get the error `AttributeError: IVCurve instance has no attribute 'v'` — Swift Arrow, Apr 04 '13 at 17:29
Thank you! That was the answer, I just didn't know how to ask! — Swift Arrow, Apr 04 '13 at 17:34

score 1 · Accepted Answer · answered Apr 04 '13 at 17:30

The problem is that you have

class IVCurve:
    def __init__(self, datetime):
        self.datetime = datetime
    v = []
    i = []
    p = []

These lists are then lists on the class. So every instance of the class has the same list. You want the lists to be different for each instance, so you should create them in your __init__ method.

class IVCurve:
    def __init__(self, datetime):
        self.datetime = datetime
        self.v = []
        self.i = []
        self.p = []

Thanks! Martijn Pieters beat you to the punch, but thanks alot anyhow! — Swift Arrow, Apr 04 '13 at 20:09

Yoriz · Answer 2 · 2013-04-04T18:12:31.707

Does this give the required output ?

import csv
import pprint


class IVCurve(object):
    def __init__(self, datetime, v=None, i=None):
        self.datetime = datetime
        self.v = []
        self.i = []
        self.p = []
        if v and i:
            self.add_data(v, i)

    def add_data(self, v, i):
        self.v.append(float(v))
        self.i.append(float(i))
        self.p.append(float(v) * float(i))

    def __repr__(self):
        return 'IVCurve (datetime: {}, v: {}, i: {}, p: {})'.format(
                self.datetime, self.v, self.i, self.p)


thelist = []

with open('Data.csv', 'rb') as ivdump:
    ivdata = csv.reader(ivdump, delimiter=';')
    datetime, v, i, p = next(ivdata)
    thelist.append(IVCurve(datetime, v, i))
    for datetime, v, i, p in ivdata:
        if datetime != thelist[-1].datetime:
            thelist.append(IVCurve(datetime, v, i))
        else:
            thelist[-1].add_data(v, i)

pprint.pprint(thelist)

Looks complicated... I have to learn more before I can understand this! I'll try it though — Swift Arrow, Apr 04 '13 at 20:12
This was very close to what I finally implemented. I accepted the other answer because it pointed out exactly what was wrong. Thanks! — Swift Arrow, Jan 01 '14 at 14:59

Parse CSV records into a list of Classes

2 Answers2

Linked