1

I'm trying to convert a pandas DataFrame to a list of dictionaries in which 1 dictionary represents 1 row; therefore pandas to_dict(orient='records') method is perfect; however the output is incorrectly rounded in some cases. Here's an example:

df = pd.DataFrame({'x': [1/3, 2/3], y=[4/3, 5/3]})
#            x         y
   0  0.333333  1.333333
   1  0.666667  1.666667

df.round(3).to_dict(orient='records')  # rounded incorrectly
# [{'x': 0.3330000000000002, 'y': 1.333}, {'x': 0.6670000000000004, 'y': 1.667}]

df.round(3).to_dict(orient='list')  # rounded correctly
# {'x': [0.333, 0.667], 'y': [1.333, 1.667]}

As you can see to_dict(orient='list') seems to work fine. What's the problem here?

Braiam
  • 1
  • 11
  • 47
  • 78
Johnny Metz
  • 5,977
  • 18
  • 82
  • 146

1 Answers1

2

In pandas 0.20.2 for some reason orient = records uses the numpy float type and the orient = list uses native python float type. 

records = df.round(3).to_dict(orient='records')
print(type(records[0]['x']))
numpy.float64

list_orient=df.round(3).to_dict(orient='list')
print(type(list_orient['x'][0]))
float

The difference in the exact data type results in the rounding difference. Now why the different orient arguments result in a different datatype, I cannot say.

When converting the numpy float back to native python float:

print(float(records[0]['x']))
0.333

We get the output like in the list oriented to_records output.

For some more info about weird float shenanigans Is floating point math broken?

error
  • 2,356
  • 3
  • 23
  • 25
  • Is this just for `0.20.2`? If I upgrade to the most recent version will this be fixed? – Johnny Metz Jul 12 '17 at 19:29
  • I have no idea if upgrading to the latest version will fix this. I explicitly mentioned in 0.20.2 since John did not this behaviour in 0.18.1. – error Jul 12 '17 at 19:31