Python RuntimeWarning: overflow encountered in long scalars

Question

I am new to programming. In my latest Python 2.7 project I encountered the following:

RuntimeWarning: overflow encountered in long_scalars

Could someone please elaborate what this means and what I could do to fix that?

The code runs through, but I'm not sure if it is a good idea to just ignore the warning.

It happens during an append process like:

SomeList.append(VeryLongFormula)

Would you please show a [short, complete example](http://sscce.org/) that demonstrates this problem? — Greg Hewgill, Sep 26 '11 at 18:36
You included the numpy tag. Nothing in your questions suggests numpy. You have not included code that allows us to reproduce the error. Please do so. — David Heffernan, Sep 26 '11 at 18:36
possible duplicate of http://stackoverflow.com/questions/3767409/python-warning-possibly-numpy — rocksportrocker, Sep 26 '11 at 18:36

unutbu · Accepted Answer · 2016-01-14T14:39:33.783

88

Here's an example which issues the same warning:

import numpy as np
np.seterr(all='warn')
A = np.array([10])
a=A[-1]
a**a

yields

RuntimeWarning: overflow encountered in long_scalars

In the example above it happens because a is of dtype int32, and the maximim value storable in an int32 is 2**31-1. Since 10**10 > 2**32-1, the exponentiation results in a number that is bigger than that which can be stored in an int32.

Note that you can not rely on np.seterr(all='warn') to catch all overflow errors in numpy. For example, on 32-bit NumPy

>>> np.multiply.reduce(np.arange(21)+1)
-1195114496

while on 64-bit NumPy:

>>> np.multiply.reduce(np.arange(21)+1)
-4249290049419214848

Both fail without any warning, although it is also due to an overflow error. The correct answer is that 21! equals

In [47]: import math

In [48]: math.factorial(21)
Out[50]: 51090942171709440000L

According to numpy developer, Robert Kern,

Unlike true floating point errors (where the hardware FPU sets a flag whenever it does an atomic operation that overflows), we need to implement the integer overflow detection ourselves. We do it on the scalars, but not arrays because it would be too slow to implement for every atomic operation on arrays.

So the burden is on you to choose appropriate dtypes so that no operation overflows.

edited Jan 14 '16 at 14:39

answered Sep 26 '11 at 19:02

unutbu

842,883
184
1,785
1,677

3

Thanks! How do I define what dtype I want? – timkado Sep 26 '11 at 19:11
13

You can set the `dtype` when creating the numpy array. For example, in my example above, you can avoid the overflow error by setting: `A = np.array([10],dtype='int64')` – unutbu Sep 26 '11 at 19:24
4

Here is a [list of basic dtypes](http://docs.scipy.org/doc/numpy/user/basics.types.html#data-types). – unutbu Sep 26 '11 at 19:26
3

Thank you very much!!! I converted the variables AF and RT into float64: `AF = np.float64(AF)` and the warning is gone. – timkado Sep 26 '11 at 19:50
1

`np.multiply.reduce(np.arange(17)+1)` doesn't evaluate to `-288522240` for me, but to the big integer you mentioned. Is it machine dependent? – Zelphir Kaltstahl Jan 14 '16 at 09:02
2

@Zelphir: Thank you for pointing this out. You are correct -- on 32-bit OSes, `np.multiply.reduce(np.arange(17)+1)` returns `-288522240` ([ideone demo](http://ideone.com/tfUbow)), but on 64-bit OSes, it returns the correct answer, `355687428096000`. I changed the example in the post above to `np.multiply.reduce(np.arange(21)+1)` which overflows on both 32-bit an 64-bit OSes. – unutbu Jan 14 '16 at 14:40

score 24 · Answer 2 · edited Dec 05 '22 at 14:15

24

An easy way to overcome this problem is to use 64 bit type

my_list = numpy.array(my_list, dtype=numpy.float64)

edited Dec 05 '22 at 14:15

Tzane

2,752
1
10
21

answered Feb 12 '18 at 08:25

Khaled

241
2
2

Python RuntimeWarning: overflow encountered in long scalars

2 Answers2

Linked