1

I am using a function that multiplies probabilities there by creating very small values. I am using decimal.Decimal module to handle it and then when the compuation is complete I convert that decimal to logofOdds using math.log module/function. But, below a certain proability python cannot convert these very small probabilities to log2 or 10 of likelyhood ratio.

I am getting ValueError: math domain error

So, I printed the value before the traceback started and it seems to be this number:

2.4876626750969332485460767406646530276378975654773588506772125620858727319570054153525540357327805722211631386444621446226193195409521079089382667946955357511114536197822067973513019098983691433561051610219726750413489309980667312714519374641433925197450250314924925500181809328656811236486523523785835600132361529950090E-366

Other small numbers like this are getting handled by math.log though in the same program: 5.0495856951184114023890172277484001329118412629157526209503867218204386939259819037402424581363918720565886924655927609161379229574865468595907661385853201472751861413845827437245978577896538019445515183910587509474989069747817303700894727201121392323641965506674606552182934813779310061601566189062725979740753305935661E-31

Is it true? any way to fix this. I know I can take the log of the probs and then sum it along the way, but when I tried to do that, it seems I have to update several places in my program - could take significant hours or days. and there is another process to convert it back to decimal.

Thanks,

everestial007
  • 6,665
  • 7
  • 32
  • 72

1 Answers1

4

If you want to take logarithms of Decimal objects, use the ln or log10 methods. Aside from a weird special case for huge ints, math.log casts inputs to float.

whatever_decimal.ln()
user2357112
  • 260,549
  • 28
  • 431
  • 505
  • Hmm, it's good to know that `decimal` module itself has `log methods` embedded in it. – everestial007 Mar 25 '18 at 19:14
  • 1
    That's not _quite_ true—`math.log` is documented to work accurately on at least ints and floats, and it _could_ work on other types, although without some kind of protocol it's hard to see how any implementation _would_ work on anything but builtins except by treating any int subclass as int and converting everything else to float, which is [what CPython does](https://github.com/python/cpython/blob/master/Modules/mathmodule.c#L1886), although with a little extra wrapper work for things that are convertible but might overflow…). – abarnert Mar 25 '18 at 19:16
  • @user235: I know it may not be a good practice, but may I direct you to another problem. I put bounty on it but it is not receiving attention. Based on your rep, I am hoping you can help. https://stackoverflow.com/questions/49429368/how-to-solve-memory-issues-problems-while-multiprocessing-using-pool-map – everestial007 Mar 25 '18 at 19:17
  • @abarnert: I was actually editing in a mention of the weird special case. I don't see any documentation for the huge int handling, though. – user2357112 Mar 25 '18 at 19:18
  • @abarnert : I though so. But, why is it not able to handle that specific number though? Other longer floats, but smaller than that are handled. Also, based on you rep, can you look into another issue. You too seem to have a good rep and I am not able to attract people on this problem too https://stackoverflow.com/questions/49429368/how-to-solve-memory-issues-problems-while-multiprocessing-using-pool-map – everestial007 Mar 25 '18 at 19:19
  • @user2357112 Hmmm… maybe the only documentation is the What's New for Python 2.1 or something similarly undiscoverable… – abarnert Mar 25 '18 at 19:19
  • 1
    @everestial007 A CPython `float` is a C `double`, which on almost every platform is an IEEE double (or newer IEEE binary64, which amounts to the same thing), and `2.4...e-366` is smaller than the smallest denormal IEEE double, so converting it to float underflows to 0.0. So you're asking for the log of 0, which is of course a domain error. – abarnert Mar 25 '18 at 19:22
  • @everestial007 More concisely, try `print(float(val))` before `print(math.log(val))` and you'll see that the first line always prints 0.0 when the second line raises a ValueError – abarnert Mar 25 '18 at 19:24
  • Good thing to know. Btw, seems like there is no `log2` implementation in `decimal` module ? – everestial007 Mar 25 '18 at 19:25
  • 1
    There is no `log2` method. You can compute it yourself as `whatever_decimal.ln() / Decimal('2').ln()`. – user2357112 Mar 25 '18 at 19:35