1

I have 3 fairly sizable dataframes that I am trying to merge in Python. I found the following technique here:

df1.merge(df2,on='name').merge(df3,on='name')

When I tried to use this on my dataframes I did not receive any errors, however each time I tried to run it I had to kill the program because PyCharm would freeze up. Is it possible that my dataframes are simply too large to merge? I have no problems merging 2 at a time so I can't imagine that that's the case, but I don't know. I really need to analyze the three together so any suggestions would be helpful.

Community
  • 1
  • 1
Sergei
  • 173
  • 1
  • 3
  • 14
  • what happens when you run directly your program and not run it through pycharm? – Zeugma Jan 20 '17 at 14:31
  • What is does the dataframe look like? – Bob Baxley Jan 20 '17 at 14:32
  • @Boud I don't really know what you mean/how to do that. Do you mean to run it through terminal? – Sergei Jan 20 '17 at 14:37
  • @BobBaxley a merging of two of the frames yields one that is [190431 rows x 12 columns]. Or are you looking for some other info? – Sergei Jan 20 '17 at 14:37
  • 1
    as an alternative you can try this: `pd.concat([df1.set_index('name'), df2.set_index('name'), df3.set_index('name')], axis=1).reset_index()` – MaxU - stand with Ukraine Jan 20 '17 at 14:38
  • @mildlyillogical yes through terminal, normal python.exe yourprogram.py – Zeugma Jan 20 '17 at 14:41
  • The other relevant pieces of info are: how big is df3? and how many repeated `name` values are there? This will tell you how big you can expect your output to be. – Bob Baxley Jan 20 '17 at 14:41
  • So I realized that part of the problem was that I was trying to merge the whole dataframes when what I really needed to do was to merge them after I grouped them by 'name'. But now I'm trying to merge just two of the new aggregate multidimensional frames and getting an error telling me that the 'name' I grouped by is not a valid key – Sergei Jan 20 '17 at 18:06
  • @mildlyillogical, please provide a sample (3-5 rows) input data sets and desired (output) data set – MaxU - stand with Ukraine Jan 20 '17 at 19:11

0 Answers0