2

I need to get qty number by a specified precision like below:

qty = 0.1284
precision = 0.01
output = 0.12

It seems easy, buy I could not get it done because I have a precision above variable.

Mark Dickinson
  • 29,088
  • 9
  • 83
  • 120
tompal18
  • 1,164
  • 2
  • 21
  • 39
  • Does this answer your question? [Limiting floats to two decimal points](https://stackoverflow.com/questions/455612/limiting-floats-to-two-decimal-points) – Tomerikoo Nov 30 '20 at 09:38
  • tompal18 Are you sure you want 0.1284 --> 0.12 and not 0.13? Rounding is more often thought of as rounding to nearest than truncation. – chux - Reinstate Monica Nov 30 '20 at 17:31
  • @chux-ReinstateMonica in that case - take in count my edit... I actually added the `Round` part so you might want to look at the original (and maybe edit my edit ^_^) – Tomerikoo Nov 30 '20 at 17:32
  • @Tomerikoo Hmm, either way, what OP is asking for is problematic is asking for truncation drive by not exact power-of-10 values like `0.01` realized in as a binary floating point. Need more input from OP. – chux - Reinstate Monica Nov 30 '20 at 17:36
  • Will the precision always be a power of 10, or are other values possible? And do you want to round or truncate? The example you gives suggests truncating, but the solution you posted uses rounding. – Mark Dickinson Dec 01 '20 at 11:23

2 Answers2

2

Floating math is not precise - see Is floating point math broken?.

You can use round() to round - you seem to want to truncate though - you can do this mathematically:

qty = 0.1284
precision = 0.01 

truncated = ((qty * (1/precision)) // 1 ) * precision

print(truncated)

Output:

0.12

This multiplies your number with the correct factor, integer divides by 1 to get rid of remaining partial digits and multiplies by precision again to get back to your float value.

Patrick Artner
  • 50,409
  • 9
  • 43
  • 69
  • 1
    Note that rounding the number in-memory may *still* lead to display imprecisions, as most floating point numbers wil remain approximations of the decimal expectation. – Masklinn Nov 30 '20 at 09:37
  • It works, but when I have qty like 181.1231231221, it prints 181.10000000000002. – tompal18 Dec 01 '20 at 01:18
0

I have done it like below.

def get_qty(qty, step_size):

    prec = 0
    if(step_size >= 1):
        return int(qty)
    while(step_size < 1):
        step_size *= 10
        prec += 1
    qty = "{:0.0{}f}".format(qty, prec)
    return float(qty)
get_qty(181.1231231221, 0.01)
get_qty(181.1231231221, 1.0)
tompal18
  • 1,164
  • 2
  • 21
  • 39
  • Doesn't this round rather than truncating? In your original post, you say you want `get_qty(0.1284, 0.01)` to return `0.12`, but this code will return `0.13`. – Mark Dickinson Dec 01 '20 at 11:22