Wrong CSV printing in Python (enumerating numpy array)

Question

I apologize if this question looks like a duplicate. I am trying to write a 7x2 array to a .csv file. The array I want to print is called x5:

x5
Out[47]: 
array([[  0.5,   1. ],
       [  0.7,   3. ],
       [  1.1,   5. ],
       [  1.9,   6. ],
       [  2. ,   7. ],
       [  2.2,   9. ],
       [  3.1,  10. ]])

The code I use:

import time
import csv
import numpy
timestr = time.strftime("%Y%m%d-%H%M%S")
with open('mydir\\AreaIntCurve'+'_'+str(timestr)+'.csv', 'w') as csvfile:
    writer = csv.writer(csvfile)
    writer.writerow(['Unique value', ' Occurrences'])
    for m, val in numpy.ndenumerate(x5):
        writer.writerow([m, val])

The result I get:

Unique value, Occurrences
"(0, 0)",0.5
"(0, 1)",1.0
"(1, 0)",0.69999999999999996
"(1, 1)",3.0 
"(2, 0)",1.1000000000000001
"(2, 1)",5.0
"(3, 0)",1.8999999999999999
"(3, 1)",6.0
"(4, 0)",2.0
"(4, 1)",7.0
"(5, 0)",2.2000000000000002
"(5, 1)",9.0
"(6, 0)",3.1000000000000001
"(6, 1)",10.0

The result I want:

Unique value, Occurrences
0.5, 1
0.7, 3
1.1, 5
1.9, 6
2.0, 7
2.2, 9
3.1, 10

I assume the problem is with ndenumerate(x5), which prints the coordinates of my values. I have tried different approaches like numpy.savetxt, but it does not produce what I want and also does not print the current date in the file name. How to amend the ndenumerate() call to get rid of the value coordinates, while keeping the current date in the file name? Thanks a lot!

It's not clear to me how you expect to get to your final result. `csv` is currently doing what you ask it to, i.e. writing out a tuple and a float separated by a comma; if that's not right, have you considered e.g. unpacking the tuple? If you want to count the appearances of each float, try `collections.Counter`. — jonrsharpe, Sep 25 '15 at 13:24
I have tried `collections.Counter` but I find myself more comfortable with my own piece of code for counting the occurrences. — FaCoffee, Sep 25 '15 at 13:27
It seems like you don't want the `i, j` at all, so why are you using `ndenumerate`? Surely you just want to pass each "row" in the array to `writerow`? — jonrsharpe, Sep 25 '15 at 13:29
You can probably just pass the whole array to `writerows`, although I haven't tried that with `csv` before. — jonrsharpe, Sep 25 '15 at 13:33

score 2 · Answer 1 · answered Sep 25 '15 at 14:05

Here's an alternative that uses numpy.savetxt instead of the csv library:

In [17]: x5
Out[17]: 
array([[  0.5,   1. ],
       [  0.7,   3. ],
       [  1.1,   5. ],
       [  1.9,   6. ],
       [  2. ,   7. ],
       [  2.2,   9. ],
       [  3.1,  10. ]])

In [18]: np.savetxt('foo.csv', x5, fmt=['%4.1f', '%4i'], header='Unique value, Occurrences', delimiter=',', comments='')

In [19]: !cat foo.csv
Unique value, Occurrences
 0.5,   1
 0.7,   3
 1.1,   5
 1.9,   6
 2.0,   7
 2.2,   9
 3.1,  10

score 1 · Answer 2 · edited May 23 '17 at 11:58

1

Have you tried replacing your two last lines of code with

for x in x5:
    writer.writerow(x)

?

You may be surpised to see 1.8999999999999999 instead of 1.9 in your csv result; that is because 1.9 cannot be represented exactly in floating point arithmetics (see this question).

If you want to limit the number of digits to 3, you can replace the last line with writer.writerow([["{0:.3f}".format(val) for val in x]])

But this will also add three zeroes to integer values. Since you can check if a float is an integer with is_integer(), you can avoid this with

writer.writerow([str(y) if y.is_integer() else "{0:.3f}".format(y) for y in x])

edited May 23 '17 at 11:58

Community

1
1

answered Sep 25 '15 at 13:27

P. Camilleri

12,664
7
41
76

@jonrsharpe Well aren't there commas in the expected output OP posted ? – P. Camilleri Sep 25 '15 at 13:29
2

No, I mean *literally every digit* - `writerow` expects a sequence, so if you pass it a string you get e.g. `'0','.','5',',',' ','1'`. See https://docs.python.org/2/library/csv.html#writer-objects – jonrsharpe Sep 25 '15 at 13:30
@jonrsharpe okay, I had tested this with print(). I'll edit the answer – P. Camilleri Sep 25 '15 at 13:32
I was getting commas all over the place, indeed. Thanks for pointing this out, @jonrsharpe! – FaCoffee Sep 25 '15 at 13:33
1

@FrancescoCastellani see the edit on how to limit the number of decimals – P. Camilleri Sep 25 '15 at 13:58

score 1 · Accepted Answer · answered Sep 25 '15 at 13:28

1

replace this line

for m, val in numpy.ndenumerate(x5):
    writer.writerow([m, val])

with:

for val in x5:
    writer.writerow(val)

you dont need to do ndenumerate

answered Sep 25 '15 at 13:28

DorElias

2,243
15
18

How do you suggest that I amend the question, to make your answer easier to find? I suspect the original question is too vague/generic and could lead to duplicates in the future due to posters overlooking it... – FaCoffee Sep 25 '15 at 13:39
1

well im not sure, perhaps mention some of the keywords "enumerating numpy array" – DorElias Sep 25 '15 at 13:41
Very quick question: what if I would like to round the values of my first column to include 3 decimals only? I see problems with numbers like `0.7`, which is printed out `0.69999999999999996` to the resulting csv file. Thanks! – FaCoffee Sep 25 '15 at 13:54
you can do `for m,val in x5` (like before only without ndenumerate) and write something like this : `writer.writerow([m,float("{0:.2f}".format(val))])` – DorElias Sep 25 '15 at 14:04
Thanks for the suggestion but sorry, I really see no difference in my output. I see the very same thing as before... – FaCoffee Sep 25 '15 at 14:11
1

well i just noticed i rounded up the second value (i assume you just copy and paste it) try doing `writer.writerow([float("{0:.3f}".format(m)), val])` – DorElias Sep 25 '15 at 14:14

Wrong CSV printing in Python (enumerating numpy array)

3 Answers3