Pandas join on these two dataframes causes value error, but different errors for different versions, and why?

Question

Possible duplicate pandas - Merging on string columns not working (bug?)

I am getting interesting messages when trying to merge the following two dataframes. Does anyone know what's going on? Removing the E fixes the error.

newvi = pd.DataFrame({'a':['E1000305', 'E1000505', 'E1001071', 'E99836', 'E1003822']})
newsm = pd.DataFrame({'a':['E1000305', 'E1000305', 'E1000305', 'E1000305', 'E1000305']})

newvi.a
#0    E1000305
#1    E1000505
#2    E1001071
#3      E99836
#4    E1003822
#Name: a, dtype: object
newsm.a
#0    E1000305
#1    E1000305
#2    E1000305
#3    E1000305
#4    E1000305
#Name: a, dtype: object

newvi.join(newsm, on='a')

#my computer
#ValueError: columns overlap but no suffix specified: Index([u'a'], dtype='object')

>>> pd.__version__
u'0.18.1'

#server

#ValueError: You are trying to merge on object and int64 columns. If you wish to proceed you should use pd.concat

>>> pd.__version__
'0.24.2'

What do you want the joined data frame to look like? – PacketLoss Jan 07 '20 at 01:51 — PacketLoss, Jan 07 '20 at 01:51

score 0 · Answer 1 · answered Jan 07 '20 at 01:57

0

newvi.join(newsm.set_index('a'), on='a')

Setting the index to 'a' seems to resolve your error, does the result match your expectation?

answered Jan 07 '20 at 01:57

PacketLoss

5,561
1
9
27

It solves it yes. But still interested why it's happening – Sam Weisenthal Jan 07 '20 at 02:06
It seems to be related to the behaviour of `on` in join and how it uses index's. https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.join.html Refer to `on` documentation. – PacketLoss Jan 07 '20 at 02:15
I think it’s a bug no – Sam Weisenthal Jan 07 '20 at 03:31

Pandas join on these two dataframes causes value error, but different errors for different versions, and why?

1 Answers1