0

I am trying to append to Panda tables, but when doing so, a both a new column and some rows are created created somehow. I do not get where it comes from. I used both the concat and append function and i saw the examples in other posts but somehow for me it does not work. This is what happens:

let us say as ab example i have df1:

"x" "y" "id"
 1   2   1
 1   3   1
 ...

and df2:

"x" "y" "id"
 5   6   2
 4   7   2
 ...

I want a table like:

"x" "y" "id"
 1   2   1
 1   3   1
 ...
 5   6   2
 4   7   2
 ...

When i use:

df=df.append(df2)

somehow i get this:

0   "x" "y" "id"
"x"  NaN  NaN NaN
"y"  NaN  NaN NaN
"id" NaN  NaN NaN
NaN   1    2   1
NaN   1    3   1
     ...
NaN   5    6   2
NaN   4    7   2
   ...

Does anyone have an idea why? My code looks like this:

columns = ["x", "y", "t", "id", "id2"]
df_allTrajectories=pd.DataFrame(columns)
for z in range(trajectory_amount):
    ....does sth
    f = open("...")
    with open("...") as f1:
        data3 = f1.read()
        TESTDATA = StringIO(data3)
    df = pd.read_table(TESTDATA, sep=" ")
    df.columns = ["x", "y", "id", "id2"]
    df.insert(loc=2, column='t', value=0)
    df.loc[:, 't'] = 0
    df.loc[:, 'id2'] = 0
    df.loc[:, 'id'] = z
    df.columns = ["x", "y", "t", "id", "id2"]
    df['id2'] =1
    df['id2'] = df.groupby('id')['id2'].cumsum().add(-1)
    df['t'] =1
    df['t'] = df.groupby('id')['t'].cumsum().add(-1)
    df_allTrajectories = df_allTrajectories.append(df)

So i am recursively appending df z times to df_allTrajectories. They both have 5 columns but somehow i get a similar wrong output like in the example above. Does anyone know why?

I also tried deleting this new first column by

df_allTrajectories.drop(df_allTrajectories.columns[[0]], axis=1) 

but without any effect.

Brian Tompsett - 汤莱恩
  • 5,753
  • 72
  • 57
  • 129
Mauritius
  • 265
  • 1
  • 8
  • 23
  • 1
    pd.concat([df1,df2]) – BENY Aug 21 '18 at 15:20
  • https://pandas.pydata.org/pandas-docs/stable/generated/pandas.concat.html – FHTMitchell Aug 21 '18 at 15:20
  • I tried concat but it happens just the same as with append – Mauritius Aug 21 '18 at 15:23
  • @FHTMitchell that is not a duplicate, concat is not working as i want it to work. i know what concat does but it seems to be a problem with my data – Mauritius Aug 21 '18 at 15:25
  • It worked fine for me with the data you provided. What error do you get? We won't debug for you. – FHTMitchell Aug 21 '18 at 15:27
  • I know that you won't debug. I do not get an error but I get an output that is quite different from how it looks when i see other examples of the `append` or `concat` functions. You can see my example above. I do not get why I get that 3 upper rows and the first column, like i explain in my example. Do you have any idea? – Mauritius Aug 21 '18 at 15:33

0 Answers0