Update pandas dataframe values obtained from a function

Question

my goal is to update my 'df1' dataframe with the new 'df2' values that are produced by a function. This is the simplification of a more complex script and I cannot exempt myself from using two functions.

import pandas as pd

def new_df2(i):
    d2 = {'col1': [i, i-1], 'col2': [i+1, i+2]}
    df2 = pd.DataFrame(data=d2)
    merge_df(df1, df2)

def merge_df(df1, df2):
    df1 = df1.append(df2)

d1 = {'col1': [5, 6], 'col2': [7, 8]}
df1 = pd.DataFrame(data=d1)

for i in range(1,3,1):
    new_df2(i)

my result:

print(df1)

   col1  col2
0     5     7
1     6     8

expected result:

print(df1)

   col1  col2
0     5     7
1     6     8
0     1     2
1     0     3
0     2     3
1     1     4

Please [provide a reproducible copy of the DataFrame with `to_clipboard`](https://stackoverflow.com/questions/52413246/how-to-provide-a-copy-of-your-dataframe-with-to-clipboard) — Trenton McKinney, May 10 '20 at 22:16

score 2 · Answer 1 · answered May 10 '20 at 22:23

Something like this would work:

In [232]: d1 = {'col1': [5, 6], 'col2': [7, 8]}                                                                                                                                                             

In [233]: df1 = pd.DataFrame(data=d1)                                                                                                                                                                       

In [234]: df = pd.DataFrame()                                                                                                                                                                               

In [235]: def new_df2(i): 
     ...:     d2 = {'col1': [i, i-1], 'col2': [i+1, i+2]} 
     ...:     df2 = pd.DataFrame(data=d2)  
     ...:     return df2  
     ...:                                                                                                                                                                                                   

In [236]: def merge_df(df1, df2): 
     ...:     df1 = df1.append(df2)  
     ...:     return df1  
     ...:                                                                                                                                                                                                   

In [237]: for i in range(1,3,1): 
     ...:     df2 = new_df2(i) 
     ...:     df = df.append(df2)  
     ...:                                                                                                                                                                                                   

In [238]: df1 = merge_df(df1,df)                                                                                                                                                                            

In [239]: df1                                                                                                                                                                                               
Out[239]: 
   col1  col2
0     5     7
1     6     8
0     1     2
1     0     3
0     2     3
1     1     4

score 1 · Accepted Answer · answered May 10 '20 at 22:25

You are having a misunderstanding regarding global and local variables for functions and I would recommend you research this topic. Functions work primarily with local variables that get passed into them. So you have to follow your dataframe in and return it out as a variable. Here is the working code and I'll comment further below.

def new_df2(i, df1):
    d2 = {"col1": [i, i - 1], "col2": [i + 1, i + 2]}
    df2 = pd.DataFrame(data=d2)
    return merge_df(df1, df2)


def merge_df(df1, df2):
    return df1.append(df2)


d1 = {"col1": [5, 6], "col2": [7, 8]}
df1 = pd.DataFrame(data=d1)

for i in range(1, 3, 1):
    df1 = new_df2(i, df1)
print(df1)
   col1  col2
0     5     7
1     6     8
0     1     2
1     0     3
0     2     3
1     1     4

Starting at the bottom you have to pass in the existing dataframe you want to modify as a variable.

for i in range(1, 3, 1):
    df1 = new_df2(i, df1)

Then in new_df2 it gets accepted as a variable here:

def new_df2(i, df1):

Then you create df2 and then send both as variables to merge_df here:

return merge_df(df1, df2)

The return will return the result of the merge after it is complete.

merge_df accepts the variables here:

def merge_df(df1, df2):
    return df1.append(df2)

And returns them to new_df2 which immediately returns it to the initial call. Then we set df1 to the new value returned, and move to the next step in the loop.

Update pandas dataframe values ​obtained from a function

2 Answers2

Update pandas dataframe values obtained from a function