append pandas dataframe automatically cast as float but want int

Question

How do I get pandas to append an integer and keep the integer data type? I realize I can df.test.astype(int) to the entire column after I have put in the data but if I can do it at the time I'm appending the data it seems like that would be a better way. Here is a sample:

from bitstring import BitArray
import pandas as pd
df = pd.DataFrame()

test = BitArray('0x01')
test = int(test.hex)
print(test)
df = df.append({'test':test, 'another':5}, ignore_index=True)

print(df.test)
print(df.another)

Here is the output:

1
0    1.0
Name: test, dtype: float64
0    5.0
Name: another, dtype: float64

It is changing the integers to floats.

Answer 1

It's because your initial dataframe is empty. Initialize it with some integer column.

df = pd.DataFrame(dict(A=[], test=[], another=[]), dtype=int)
df.append(dict(A=3, test=4, another=5), ignore_index=True)

---

Had I done

df = pd.DataFrame()
df.append(dict(A=3, test=4, another=5), ignore_index=True)

Answer 2

You need to use convert_dtypes, if you are using Pandas 1.0.0 and above. Refer link for description and use convert_dtypes

df = df.convert_dtypes()
df = df.append({'test':test, 'another':5}, ignore_index=True)

Answer 3

As in this issue: df.append should retain columns type if same type #18359, append method will retain column types since pandas 0.23.0.

So upgrading pandas version to 0.23.0 or newer solves this problem.

Answer 4

Well there are 2 workarounds, I found.

1. Upgrade to pandas version >= 0.23.0

2. But if one cannot change pandas version like when working for production code and version change might affect other scripts/codes in prod environment. so below one-liner is a quick workaround.

df = df.astype(int)