2

I have two dataframes with the same headers. When I try to concatenate (pd.concat()) these two dfs (df1, df2), I get the error:

"InvalidIndexError: Reindexing only valid with uniquely valued Index objects"

I figured out that the problem is that there are duplicate column names within each dataframe. Example:

df1

respondent ID - Column1 - Column2 - Column3 - Column1 - Column2 - Column3

df2

respondent ID - Column1 - Column2 - Column3 - Column1 - Column2 - Column3

in theory, I just want to add the data of df2 below df1 (without the headers of df2 ofc)

How do I bypass this? Any thoughts?

mozway
  • 194,879
  • 13
  • 39
  • 75
MikeRuud
  • 23
  • 1
  • 4
  • Does this answer your question? [Pandas: append dataframe to another df](https://stackoverflow.com/questions/39815646/pandas-append-dataframe-to-another-df) – Thekingis007 Nov 11 '21 at 14:05

2 Answers2

2

The duplicated columns shouldn't be an issue (even with ignore_index=False):

df1 = pd.DataFrame([range(7)], columns=['respondent ID', 'Column1', 'Column2', 'Column3', 'Column1', 'Column2', 'Column3'])
df2 = pd.DataFrame([['2']*7], columns=['respondent ID', 'Column1', 'Column2', 'Column3', 'Column1', 'Column2', 'Column3'])
pd.concat([df1, df2], ignore_index=True)

output:

  respondent ID Column1 Column2 Column3 Column1 Column2 Column3
0             0       1       2       3       4       5       6
1             2       2       2       2       2       2       2
mozway
  • 194,879
  • 13
  • 39
  • 75
1

You can use this solution:

df1 = df1.append(df2, ignore_index=True)
Thekingis007
  • 627
  • 2
  • 16