Write data to file

Question

I am trying to write a matrix (<type 'numpy.ndarray'>) in a file, with this format:

index_of_row # v0, v1, v2

which will be read by my partner's Scala code (if that matters).

After reading this, I ended up with this code:

print dense_reduced
# this will give an error:
#TypeError: expected a single-segment buffer object
#f = open('dense.txt','w')
#f.write(dense_reduced[0])
#f.close()
numpy.savetxt('dense.txt', dense_reduced, delimiter=", ", fmt="%s") 

which outputs:

[[-0.17033304  0.13854157  0.22427917]
..
 [-0.15361054  0.38628932  0.05236084]]

and the dense.txt is:

-0.170333043895, 0.138541569519, 0.224279174382
...

However, there are several reasons I need the dense.txt to look like this (index of row of the matrix # values separated by comma):

0 # -0.17033304,  0.13854157,  0.22427917
...

How to proceed?

Answer 1

With savetext() options :

u = dense_reduced
w = np.hstack((np.arange(u.shape[0]).reshape(-1,1),u))
np.savetxt('dense.txt', w, fmt=["%i #"]+ ["%.10s, "]*(u.shape[1]-1)+["%.10s"])

for :

0 # 0.57105063,  0.70274226,  0.87870916
1 # 0.28735507,  0.94860021,  0.63763897 
2 # 0.26302099,  0.26609319,  0.75001683 
3 # 0.93315750,  0.19700358,  0.13632004

You can also simplify with w=pd.DataFrame(u).reset_index() if you have pandas.

Answer 2

There are several options that you can provide in numpy.savetxt (such as comments, delimiter, etc) but I don't believe you can do it in this way. A multidimensional np array can be used as an iterable of smaller arrays, so we can easily run:

my_array = np.array(range(20)).reshape((4,5))
f = open("output.txt", "w")
for i, a in enumerate(my_array):
    f.write("{} # {}".format(i, ', '.join(list(map(str, a)))))
f.close()