0

I am trying to write a script that takes csv data dumps from analog sensors, with csvs of varying lengths, and reads them into a single dimension array. The sensor samples at 32 samples/sec, so each horizontal ROW is a buffer of 32 values (1 second of data). I need all of these values, from eveny N row, in a single dimension array that I can then plot as a Y axis with matplotlib.

Here is my most recent attempt in python 2.7x. I have managed to get multiple 1D arrays, but each line is a different array, which doesn't work.

NOTE: I know how to join arrays manually, but these could be 10s, 100s, or 1000s of rows long, so I obviously need to automate this process. I am would prefer to achieve this all with the standard library and/or numpy ONLY if possible. Thanks in advance.

fo = open("data.csv", "r")
lines = fo.readlines()
for line in lines:
    value = line.split(",")
    result = map(float, value)
    nparray = np.asarray(res)
    yAxis = npa.flatten()
fo.close()

An 'example' input CSV file might look like:

1,2,3,4,5,6,7,8,9,10 \n

11,12,13,14,15,16,17,18,19,20 \n

21,22,23,24,25,26,27,28,29,30 \n

etc.. (and on and on for an UNDETERMINED amount of rows)

output:

[1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]

The numpy array conversion flattens the 2d array to multiple 1d arrays. The issue from there is not being able to join an unknown amount of arrays. I also realize there must be a MUCH more efficient way of doing this task in Python.

TigerhawkT3
  • 48,464
  • 6
  • 60
  • 97
J. Stevenson
  • 9
  • 3
  • An example of expected input / output would help. – shad0w_wa1k3r Mar 08 '17 at 07:01
  • How about `yAxis.append()`? – Sangbok Lee Mar 08 '17 at 07:03
  • 1
    Briefly: `import numpy as np; result = np.genfromtxt('data.csv', delimiter=',').flatten()`. – TigerhawkT3 Mar 08 '17 at 07:03
  • @TigerhawkT3 that does exactly what I need, thank you. The suggested duplicate does not match your comment here though. I would like to mark your solution as the correct one if possible, but I don't think I can mark a comment as the correct solution. I am new here, so please advise on the correct protocol from here. Thanks. – J. Stevenson Mar 08 '17 at 07:26
  • There are two linked duplicates in the list, not just one. I got `np.genfromtxt('data.csv', delimiter=',')` from the "How to read csv..." question, and `.flatten()` from the "correct and efficient way to..." question. – TigerhawkT3 Mar 08 '17 at 08:41

0 Answers0