How to efficiently output n items per line from numpy array

Question

I've got a 1-D numpy array that is quite long. I would like to efficiently write it to a file, putting N space separated values per line in the file. I have tried a couple of methods, but both have big issues.

First, I tried reshaping the array to be N columns wide. Given a file handle, f:

myArray.reshape(-1, N)
for row in myArray:
    print >> f, " ".join(str(val) for val in row)

This was quite efficient, but requires the array to have a multiple of N elements. If the last row only contained 1 element (and N was larger than one) I would only want to print 1 element... not crash.

Next, I tried printing with a counter, and inserting a line break after every Nth element:

i = 1
for val in myArray:
    if i < N:
        print >> f, str(val)+" ",
        i+=1
    else:
        print >> f, str(val)
        i = 1

This worked fine for any length array, but was extremely slow (taking at least 10x longer than my first option). I am outputting many files, from many arrays, and can not use this method due to speed.

Any thoughts on an efficient way to do this output?

score 3 · Accepted Answer · answered Sep 23 '14 at 10:07

3

for i in range(0, len(myArray), N):
    print " ".join([str(v) for v in myArray[i:i+N]])
    # or this 
    # print " ".join(map(str, myArray[i:i+N].tolist()))

answered Sep 23 '14 at 10:07

cronos

2,268
16
17

score 0 · Answer 2 · answered Sep 23 '14 at 10:20

You could add an try/except to your reshaping approach to print the last elements to the output file:

myArray.reshape(-1, N)
try:
    for row in myArray:
        print >> f, " ".join(str(val) for val in row)
except: # add the exact type of error here to be on the save side
    # just print the last (incomplete) row
    print >> f, " ".join(str(val) for val in myArray[-1])

How to efficiently output n items per line from numpy array

2 Answers2

Linked