0

I'm attempting to append one DataFrame to the end of another. The first DataFrame has 100 columns. The second DataFrame has 30 columns. (The second's columns are a subset of the first's.) My intention is any column not present in the second will simply be assigned an NA value.

I'm following the example described in the Pandas documentation:https://pandas.pydata.org/pandas-docs/stable/merging.html#concatenating-using-append

The second example shows that if the two DataFrames have different columns, pandas will intelligently fill the missing data with NA.

However, when I try this, I just get: "ValueError: Plan shapes are not aligned"

    df_a.shape
    Out[27]: (50000, 100)

    df_b.shape
    Out[26]: (414, 30)

    df_all = df_a.append(df_b, ignore_index=True)

Can anybody help explain what I may be doing wrong?

Thank You!

Noah
  • 567
  • 8
  • 19

1 Answers1

0

Although I can't be 100% sure (since you haven't posted much about your data), I came across the same error while using append and it turned out to be that one of the dataframes had duplicated column names. Once I corrected this the error went away.

It is a similar issue discussed in the following two posts: concatenate dataframes with different column ordering and Pandas concat gives error ValueError: Plan shapes are not aligned

but in the context of pd.concat in those posts (although it seems the issues are the same)

Byran
  • 36
  • 3