I have several empty dataframes, all of different lengths, they all share some common fields, but each dataframe has some unique columns as well. In trying to concatenate them I keep getting a ValueError: Plan shapes are not aligned
.
cols1 = ['Test', 'Col1', 'Col2']
cols2 = ['Test', 'Col1', 'Col2', 'Date']
cols3 = ['Test', 'Col1', 'Col2', 'Time']
df1 = pd.DataFrame(columns=cols1)
df2 = pd.DataFrame(columns=cols2)
df3 = pd.DataFrame(columns=cols3)
df_dict = {'project': [df1, df2, df3]}
for table, dfs in df_dict.items():
result = pd.concat(dfs)
Ideally my output should have a columns list that looks like this:
['Test1', 'Col1', 'Col2', 'Date', 'Time']
Is there any way to get past this without using Merge
. I don't want to use Merge
because I won't always know the columns that are shared, etc...