Merging(?) Smaller DataFrame into bigger DataFrame (Pandas/Python)

Asked Feb 13 '19 at 21:07

Active Feb 13 '19 at 21:07

Viewed 15 times

I am running a process where I am trying to merge some predicted values back into another dataframe which has the 'actual' values.

The problem I'm having is that after the merge is done, I end up with a new dataframe that only has 1 row. In this case, I am wanting to update the predicted_sales column in the parent dataframe to the value in yhat if the date matches.

Is it possible to take a smaller dataframe (in this case, a dataframe with 1 record) and marge it into a larger dataframe (n records) without having to go through and iterate on every row and see if a value should be upated?

I thought I was on the right path with pd.merge but it's not looking right:

result_df = pd.merge(orig_df, sales_forecast_df, on='date')

asked Feb 13 '19 at 21:07

mwilson

12,295
7
55
95

1

try passing `how='outer'` in your `merge` – roganjosh Feb 13 '19 at 21:09
result_df = pd.merge(orig_df, sales_forecast_df, on='date',how='left') – BENY Feb 13 '19 at 21:09
1

Looks like there is no match between date values of both dataframes, outer merge will keep all rows but introduce NaNs where it does not find corresponding value – Vaishali Feb 13 '19 at 21:09
@roganjosh that did it! – mwilson Feb 13 '19 at 21:12
Just like @vaishali said, I do end up with NaN in the rest of the values. In my case, that's fine though. – mwilson Feb 13 '19 at 21:12
I've re-read and I might be unclear on what you're trying to do. You only ever want that single item from the original df to be updated? Merge is _really_ expensive for that unless `outer` is the kind of thing you were looking for – roganjosh Feb 13 '19 at 21:12
1

Ah nm, we got there then :) – roganjosh Feb 13 '19 at 21:12
@roganjosh will mark as answer if you make your comment an answer – mwilson Feb 13 '19 at 21:14

Merging(?) Smaller DataFrame into bigger DataFrame (Pandas/Python)

0 Answers0