37

By default the numerical values in data frame are stored up to 6 decimals only. How do I get the full precision.

For example
34.98774564765 is stored as 34.987746. I do want the full value.

and 0.00000565 is stored as 0. .

Apart from applying formats to each data frame is there any global setting that helps preserving the precision.

Thanks

Sheldon
  • 1,215
  • 1
  • 15
  • 29
user7789097
  • 433
  • 1
  • 5
  • 6

3 Answers3

77

No, 34.98774564765 is merely being printed by default with six decimal places:

>>> pandas.DataFrame([34.98774564765])
           0
0  34.987746

The data itself has more precision:

>>> pandas.DataFrame([34.98774564765])[0].data[0]
34.98774564765

You can change the default used for printing frames by altering pandas.options.display.precision.

For example:

>>> pandas.set_option("display.precision", 8)
>>> pandas.DataFrame([34.98774564765])
                0
0  34.98774564765
donkopotamus
  • 22,114
  • 2
  • 48
  • 60
  • That really helped, thanks! – gies0r Nov 28 '19 at 22:18
  • Just what I was looking for - thank you. df.round(10) did not work and all other format functions did not work, too. This was perfect & simple. – 44r3n54n Mar 07 '20 at 02:57
  • This option will sometimes print things in scientific notation. Use the `display.precision` option from the other answer if you need uniform outputs – crypdick Jul 26 '21 at 22:50
  • It seems to me that `astype` is also affected, so that `column.astype(str)[0]` will be truncated – Eric Burel Oct 05 '21 at 12:29
  • I've solved it by doing `c.astype(np.float64).astype(str)`, otherwise a direct conversion from `float32` to string seems to do a kind of "display" conversion and truncates the number – Eric Burel Oct 05 '21 at 12:57
  • @Eric Burel … if your data is `float32` then it will only have around 7 significant digits of precision. Upcasting it to `float64` won’t have added any **real** information at all – donkopotamus Oct 06 '21 at 04:16
  • The weird thing is that the data seems to actually have float64 information, as if the data were right, but the "metadata" were wrong (wrongly indicating float32 and thus confusing downstream methods like `astypes`). It's in the context of loading big h5 files in pandas so I still need to double check the dataframe creation process, I didn't manage to reproduce in a unit test. – Eric Burel Oct 06 '21 at 05:49
38

You can also use the 'display.float_format' option

with pd.option_context('display.float_format', '{:0.20f}'.format):
    print(pd.DataFrame([34.98774564765]))

                        0
0 34.98774564765000150146
piRSquared
  • 285,575
  • 57
  • 475
  • 624
11

Your data is stored with the precision, corresponding to your dtype (np.float16, np.float32, np.float64).

pd.options.display.precision - allows you to change the precision for printing the data

MaxU - stand with Ukraine
  • 205,989
  • 36
  • 386
  • 419