indexing arrays in for glob loop

Question

I'm trying to create matlab files from multiple files in a directory:

import os
from glob import glob
from numpy import*
from pandas import*
import scipy.io as sio

path = '/Volumes/Some file'

for filename in glob(os.path.join(path, '*.csv')):
    df = read_csv(filename, delimiter = ';',...)
    b= df.values #make dataframe into array
    val = arange(8,13) # values with nan
    b2 = delete(b,[val], axis = 1) #remove the nan values
    c = sio.savemat('readcsv', dict(b=b2)) #convert to matlab

so what happens is that b2 in the loop when printed gives several arrays created from the files in the working directory. The problem is I don't know how to index each of the separate arrays outputted in the for loop, to create separate mat files. I tried numpy.ndenumerate:

D = array(b2)    
   for index, x in ndenumerate(b2):
       print(index,x)

but this just treats the looped b2 as one big array and indexes them separately. I want to treat each of the arrays in b2 as a subarray, and create different mat. file for each subarray. I'm not sure which method to use.

Are you asking how to create individually numbered output filenames instead of using `'readcsv'` each time? — mkrieger1, Apr 06 '18 at 19:39
No, I need to use 'readcsv' multiple times, so that it can read all my csv files in my cwd. That part of the code I think is alright, the problem I'm having is the last two lines in the code. What I need is multiple matlab files from the different arrays in b2. — Kimi Lee, Apr 06 '18 at 19:43
What you describe seems to be what I meant, so anyway... check out out the `enumerate` function. — mkrieger1, Apr 06 '18 at 19:45
Possible duplicate of [What does enumerate mean?](https://stackoverflow.com/questions/22171558/what-does-enumerate-mean) — mkrieger1, Apr 06 '18 at 19:46
To save each `b` to a different file, you need to construct a new name, either from an index or counter, or from the `filename` string. `savemat` can save several arrays, so an alternative to is to collect all the `b` in a dictionary, and do the `savemat` once, outside of the loop. — hpaulj, Apr 06 '18 at 19:58
You didn't try `enumerate`, you tried `numpy.ndenumerate`. That's something different... — mkrieger1, Apr 06 '18 at 20:00
Please pay attention to spelling in programming. I believe you also misunderstood my first comment because you confused the function `read_csv` with the filename `'readcsv'`. — mkrieger1, Apr 06 '18 at 20:00

score 0 · Accepted Answer · answered Apr 06 '18 at 21:13

As suggested by @hpaulj in the comments. Collected all the b2 arrays into one dictionary and exported as one mat file:

import os
from glob import glob
from numpy import*
from pandas import*
import scipy.io as sio

path = '/Volumes/Some file'
D_values = {}

for filename in glob(os.path.join(path, '*.csv')):
    df = read_csv(filename, delimiter = ';',...)
    b= df.values #make dataframe into array
    val = arange(8,13) # values with nan
    b2 = delete(b,[val], axis = 1) #remove the nan values
    D_values = [b2
                 for i in arange(1,121)] # there are 120 files

c = sio.savemat('readcsv', dict(b=D_values)) #convert to matlab

This doesn't export to different matlab files though, only one with a multidimensional array.

indexing arrays in for glob loop

1 Answers1