0

I have two pandas dataframes and some of the values overlap and I'd like to append to the original dataframe if the time_date value and the origin values are the same.

Here is my original dataframe called flightsDF which is very long, it has the format:

year    month   origin  dep_time   dep_delay    arr_time    time_hour
2001    01       EWR       15:00       15         17:00     2013-01-01T06:00:00Z

I have another dataframe weatherDF (much shorter than flightsDF) with some extra infomation for some of the values in the original dataframe

origin  temp    dewp    humid   wind_dir    wind_speed  precip  visib   time_hour
0   EWR     39.02   26.06   59.37   270.0   10.35702    0.0     10.0    2013-01-01T06:00:00Z
1   EWR     39.02   26.96   61.63   250.0   8.05546     0.0     10.0    2013-01-01T07:00:00Z
2   LGH     39.02   28.04   64.43   240.0   11.50780    0.0     10.0    2013-01-01T08:00:00Z

I'd like to append the extra information (temp, dewp, humid,...) from weatherDF to the original data frame if both the time_hour and origin match with the original dataframe flightsDF

I have tried

for x in weatherDF:
    if x['time_hour'] == flightsDF['time_hour'] & flightsDF['origin']=='EWR':
        flights_df.append(x)

and some other similar ways but I can't seem to get it working, can anyone help?

I am planning to append all the corresponding values and then dropping any from the combined dataframe that don't have those values.

emma.lucy33
  • 1
  • Why not simply `merge`? To be clear, adding columns is not appending data (i.e., stacking data frames) but merging data. – Parfait Mar 26 '22 at 15:08

1 Answers1

0

You are probably looking for pd.merge:

flightDF = flightsDF.merge(weatherDF, on=['origin', 'time_hour'], how='left')
print(out)

# Output
   year  month origin dep_time  dep_delay arr_time             time_hour   temp   dewp  humid  wind_dir  wind_speed  precip  visib
0  2001      1    EWR    15:00         15    17:00  2013-01-01T06:00:00Z  39.02  26.06  59.37     270.0    10.35702     0.0   10.0

If I'm right take the time to read Pandas Merging 101

Corralien
  • 109,409
  • 8
  • 28
  • 52