0

My issue is little bit strange. I have an excel file in which some float values are mentioned e.g. 0.0079 but when I try to read it using my code it's giving me 0.007899999999999999. I'm not sure what's actually happening here(may be that's the original data which was entered in the excel sheet when it was originally created). I checked other questions on SO which are similar but they are all concerned about 'how to not loose the precision' and my case is opposite.

sample code:

sheet_hdl = open_workbook('file.xls')
sheet = sheet_hdl.sheet_by_index(0)
for row_idx in range(1, sheet.nrows):
    for col_idx in range(1):
        if(val == str(sheet.cell(row_idx,col_idx + 1).value)):
            accuracy_val = repr(sheet.cell(row_idx,col_idx + 7).value)
            print(accuracy_val)

Note: It's not a duplicate of this. The question in this link involves computation, but my data in excel doesn't involve any computations.

Community
  • 1
  • 1
manty
  • 287
  • 5
  • 19
  • The above link involves computation, that's something processor is doing, but in my case it's simply the data to be read in excel. No computations involved at all. – manty Jan 04 '16 at 12:26
  • You aren't getting "added precision". You're simply seeing the way floating-point math works in binary. Just like some numbers repeat in base 10 (e.g. ⅓ = 0.333…), some other numbers repeat in base 2. – ChrisGPT was on strike Jan 04 '16 at 12:26
  • Does the cell contain a formula? In that case _Excel_ is doing the computation for you, and might display the result differently from how it is stored. – ChrisGPT was on strike Jan 04 '16 at 12:28
  • Exactly, that's my concern, there's no formula involved. – manty Jan 04 '16 at 12:29
  • 1
    Input 0.0079 [here](http://www.h-schmidt.net/FloatConverter/IEEE754.html) for example. Btw, in fact you *are* losing precision; 0.00790000000000000000... cannot be represented exactly in a 32-bit float, so the next best match is taken that does fit in 32 bits. – JimmyB Jan 04 '16 at 12:33
  • You beat me to it, Hanno. @manty, 0.0079 in decimal _cannot be represented precisely in binary_. – ChrisGPT was on strike Jan 04 '16 at 12:34
  • 1
    Another example: What output do you get when you do something like `x = 0.0079; print(x);`? – JimmyB Jan 04 '16 at 12:38
  • What if you use `Decimal`? First `from decimal import *` and then `accuracy_val = Decimal(repr(sheet.cell(row_idx,col_idx + 7).value))` – vrs Jan 04 '16 at 13:07
  • @HannoBinder doing `x=0.0079` and then `print(x)` is giving exactly `0.0079` in IDLE. – manty Jan 05 '16 at 06:02
  • @vrs Applying your method is giving error as: `decimal.InvalidOperation: []` – manty Jan 05 '16 at 06:03

0 Answers0