Save a list of array of different length to csv

Question

I have data_pressure, that's a list of numpy arrays such as:

array([ 268752.,  26222.,  261152.,  260958.]),
array([ 123433., 98239., 98932.]),
array([ 2893789., 872398., 92839., 9283982., 7632762., 547627.,])

every array has different length.

I've used Python's csv module to save to file this list of array:

import csv

csvfile = "csvfile.csv"

with open(csvfile, "w") as output:
   writer = csv.writer(output, lineterminator='\n')
   for line in data_pressure:
       writer.writerow(line)   

Everything works like a charm, but as I read it with

import csv
data = []
with open('csvfile.csv', 'rb') as csvfile:
    spamreader = csv.reader(csvfile)
    for row in spamreader:
        data.append(np.asarray(row))

I get

array([ '268752.0',  '26222.0',  '261152.0',  '260958.0']),
array([ '123433.0', '98239.0', '98932.0']),
array([ '2893789.0', '872398.0', '92839.0', '9283982.0', '7632762.0', '547627.0',])

Then every values in any array is a string type and not a float.

Is there a way to circumvent this problem?

Answer 1

You can pass quoting=quote.QUOTE_NONNUMERIC like so:

spamreader = csv.reader(csvfile, quoting=quote.QUOTE_NONNUMERIC)

See the docs:

Instructs writer objects to quote all non-numeric fields.

Instructs the reader to convert all non-quoted fields to type float.

Answer 2

A csv is a text document a is therefore going to store numeric values as character strings. Once you load the array into a numpy array object, you can convert the type of the array to float.

You can convert it from an array of strings to an array of floats using this method.

x = np.array(['1.1', '2.2', '3.3'], dtype='|S4')
y = x.astype(np.float)

Answer 3

You can modify the reading part of code as follows.

import csv
data = []    
with open('csvfile.csv', 'rb') as csvfile:
    spamreader = csv.reader(csvfile)`enter code here`
    for row in spamreader:
        data.append(np.asarray(row).astype('float64'))      #Modified line

Thank you.