How to iterate through two data frames in python

Question

I have two Data frames df1( having columns C1,C2,etc) and df2(having columns S1,S2,etc)
I want to iterate through each column of both the Data Frames.
Currently I am doing the following thing:

df3=pd.Dataframe([])
for index1,row1 in df1.iterrows():
    for index2,row2 in df2.iterrows():
        if row1['C1']==row2['S1']:
            #perform Some Operations on each row like:
            df3 = df3.append(pd.DataFrame({'A': row2['S1'], 'B': row2['S2'],'C':functionCall(row1['c3'], row2['S3'])}, index=[0]), ignore_index=True)

This works ok but it takes too much time.
I wanted to know, Is there a more efficient way of iterating through two Data Frames?

Have you tried MERGE? https://stackoverflow.com/questions/41463119/join-two-dataframes-on-common-column-in-python — Dave Anders, Jun 24 '18 at 18:06
the function call is fuzzy search `fuzz.token_set_ratio(row1['C3'], row2['S3'])` — konkun, Jun 25 '18 at 12:13
I want to find `fuz.token_set_ratio()` for each row of first dataframe with that of 2nd data frame if the `if ` condition is satisfied.Please help..i am new to python coding — konkun, Jun 25 '18 at 12:16

score 1 · Answer 1 · edited Jun 24 '18 at 19:23

1

I think need merge first, then apply function and last filter columns by subset - [[]]:

df3 = pd.merge(df1, df2, left_on='C1', right_on='S1')
df3['C'] = df3.apply(lambda x: functionCall(x['C3'], x['S3']), axis=1)
df3 = df3[['S1', 'S2', 'C']].rename(columns={'S1': 'A','S2': 'B'})

edited Jun 24 '18 at 19:23

timodriaan

55
3
8

answered Jun 24 '18 at 18:12

jezrael

822,522
95
1,334
1,252

@Momel - Thank you. – jezrael Jun 24 '18 at 19:24

How to iterate through two data frames in python

1 Answers1