Python; how to convert a string to an int when the int is too big?

Question

I have a panda series in which each columns are "objects", I want to convert some of them into integers. It does not work for columns with too large numbers, I get the error "int too big to convert".

I've searched through the web and the forum but I did not understand the various answers, I am pretty new to Python. I get that my "numbers" are too long...I don't get why Python has an issue with that... exporting the dataframe into an excel file for instance works just fine and then I can do my pivot tables without any issue (excel recognise the numbers as integers) but I really want to be able to do that in Python...

here the code

ex_numb = 2461401487610497195867
np.uint64(ex_numb)

---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
<ipython-input-109-e8c58111a107> in <module>
      1 ex_numb = 2461401487610497195867
----> 2 np.uint64(ex_numb)

OverflowError: int too big to convert

import sys sys.maxsize

9223372036854775807

For your given code example, ex_numb is already an int. But you can also just use "int(ex_numb)" to make an int out of a string. — Kim Tang, Sep 24 '20 at 09:25
Are you on python 2 or 3? Read this answer: https://stackoverflow.com/a/2104947/1116508 — Nicolai Lissau, Sep 24 '20 at 09:28
you're right... i put it as an example, but my real data are of type ... anyway it does not change the issue — Melwhowanttolearn, Sep 24 '20 at 09:28

ronkov · Answer 1 · 2020-09-28T11:48:21.820

0

It seems the issue is using np.uint because it can only handle up to 64 bits, you can use np.int instead:

import numpy as np
ex_numb = 2461401487610497195867
ex_np = np.int(ex_numb)
ex_np.bit_length()

Which outputs:

To use it in pandas, you can pass it as a numpy array:

pd.DataFrame(np.array([[ex_numb]]))

edited Sep 28 '20 at 11:48

answered Sep 24 '20 at 09:30

ronkov

1,263
9
14

indeed in that example using np.int(ex_numb) works, long(ex_numb) too but not in my dataframe... – Melwhowanttolearn Sep 24 '20 at 09:40

score 0 · Answer 2 · answered Sep 24 '20 at 10:06

I think it might be clearer with this code to understand my issue:

import pandas as pd

data = {'value':  ['156155486292071842853', '156155486292071842855','156155486292071842854'],
        'unit': ['CDC', 'OAB','BK'],
        'decim_value' : ['18','0','6']
        }

df = pd.DataFrame (data, columns = ['value','unit','decim_value'])
print (df)
df.info()

from functools import partial
df.apply(partial(pd.to_numeric, errors='ignore')).info()

It does not convert my "values"!

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 3 entries, 0 to 2
Data columns (total 3 columns):
 #   Column       Non-Null Count  Dtype 
---  ------       --------------  ----- 
 0   value        3 non-null      object
 1   unit         3 non-null      object
 2   decim_value  3 non-null      int64 
dtypes: int64(1), object(2)
memory usage: 200.0+ bytes

Python; how to convert a string to an int when the int is too big?

2 Answers2