Finding and plotting data in a csv

Question

My current code looks for this type of data:

AHR2,231,123,5,12,51
GPS,12,312,512,35,12
AHR2,13,125,125123,152,12
CMD,123,123,5,123,51,12
PRAM,1231,CM,12
PRAM,12345,DM,14
AND SO ON

Here is how I am doing it right now:

import csv
import matplotlib.pyplot as plt
AHR2 = []
CM = []
with open('data2.csv', 'r') as csvfile:
    content = csv.reader(csvfile, delimiter=',')
    for row in content:
        if 'AHR2' == row[0]:
            AHR2.append([row[0]] + list(map(float, row[1:])))
AHR2 = list(zip(*AHR2))
ax.plot(AHR2[1], AHR2[5], label='AHR2 Alt', color = 'red')

This is working fine, except when I am trying to parse the PRAM data part. As you can see there is more than 1 PRAM with different values (CM,DM) I am only interested in the ones that have CM. I have tried this but it gave me an index error.

if 'CM' == row[2]:
    CM.append([row[0]] + list(map(float, row[1:])))

Is there a way for me to look between the rows and pull the entire row like I am doing above?

Where did you add this code? It seems to be using variables you never initialized (like `CM`). — Scott Hunter, Oct 26 '16 at 18:03
I added it under if AHR2 == 0. I did have a CM initialized I just forgot to add it there when I shrunk my code to post it on here. — Mike Jones, Oct 26 '16 at 18:12

score 1 · Accepted Answer · edited May 23 '17 at 10:34

I would really suggest using pandas for this type of work, it will save you a lot of headaches. Your data is a bit messy since it doesn't have a constant number of columns per row, one way to deal with this is discussed here.

Something like this should work

import pandas as pd
MAX_COLS_PER_ROW = 7
my_cols = range(0, MAX_COLS_PER_ROW)
df = pd.read_csv('test.csv', names=my_cols)
df

Out[1]:
      0      1    2       3      4     5     6
0  AHR2    231  123       5   12.0  51.0   NaN
1   GPS     12  312     512   35.0  12.0   NaN
2  AHR2     13  125  125123  152.0  12.0   NaN
3   CMD    123  123       5  123.0  51.0  12.0
4  PRAM   1231   CM      12    NaN   NaN   NaN
5  PRAM  12345   DM      14    NaN   NaN   NaN

Then filtering can easily be done with something like

(df.loc[:,0] == "PRAM") & (df.loc[:,2] == "CM")
Out[2]:
0    False
1    False
2    False
3    False
4     True
5    False
dtype: bool

df.loc[(df.loc[:,0] == "PRAM") & (df.loc[:,2] == "CM"), :]
Out[3]:
      0     1   2   3   4   5   6
4  PRAM  1231  CM  12 NaN NaN NaN

I'll leave the plotting to you since the question seems more to do with filtering the data and I am slightly confused on the exact end goal.

score 0 · Answer 2 · answered Oct 26 '16 at 19:09

You only need to check for CM when 'PRMA' == row[0]; if you are checking row[2] for a non-PRAM line that only has 2 values, that could cause an index error.

Given that you have neither identified the line of data that is causing the problem, nor the line of code w/ the error, that's the best I can do.

Finding and plotting data in a csv

2 Answers2