I am trying to append to Panda tables, but when doing so, a both a new column and some rows are created created somehow.
I do not get where it comes from. I used both the concat
and append
function and i saw the examples in other posts but somehow for me it does not work.
This is what happens:
let us say as ab example i have df1
:
"x" "y" "id"
1 2 1
1 3 1
...
and df2
:
"x" "y" "id"
5 6 2
4 7 2
...
I want a table like:
"x" "y" "id"
1 2 1
1 3 1
...
5 6 2
4 7 2
...
When i use:
df=df.append(df2)
somehow i get this:
0 "x" "y" "id"
"x" NaN NaN NaN
"y" NaN NaN NaN
"id" NaN NaN NaN
NaN 1 2 1
NaN 1 3 1
...
NaN 5 6 2
NaN 4 7 2
...
Does anyone have an idea why? My code looks like this:
columns = ["x", "y", "t", "id", "id2"]
df_allTrajectories=pd.DataFrame(columns)
for z in range(trajectory_amount):
....does sth
f = open("...")
with open("...") as f1:
data3 = f1.read()
TESTDATA = StringIO(data3)
df = pd.read_table(TESTDATA, sep=" ")
df.columns = ["x", "y", "id", "id2"]
df.insert(loc=2, column='t', value=0)
df.loc[:, 't'] = 0
df.loc[:, 'id2'] = 0
df.loc[:, 'id'] = z
df.columns = ["x", "y", "t", "id", "id2"]
df['id2'] =1
df['id2'] = df.groupby('id')['id2'].cumsum().add(-1)
df['t'] =1
df['t'] = df.groupby('id')['t'].cumsum().add(-1)
df_allTrajectories = df_allTrajectories.append(df)
So i am recursively appending df
z
times to df_allTrajectories
. They both have 5 columns but somehow i get a similar wrong output like in the example above. Does anyone know why?
I also tried deleting this new first column by
df_allTrajectories.drop(df_allTrajectories.columns[[0]], axis=1)
but without any effect.