What would be a good way to fill the missing NAs in a dataframe column with intermediate values that grow gradually from the last non-NA value to the next non-NA value?
Here is an example: for the column cost, I would like to obtain the column cost_esti where the cost increase by $31 each year between 2014 and 2016, bridging the last known cost of $595 to the next known cost of $720
The code I came up with is lengthy. Is there an elegant way to do the same?
library(data.table)
data = data.table(year=2000:2018,
cost = c(100,120,NA,200,220,NA,NA,300,350,470,500,NA,NA,595,NA,NA,NA,720,800))
data[,cost_nas:=as.numeric(is.na(cost))]
## consecutive nas so far for each row:
data[, consecutive_nas_so_far := seq_len(.N), by=rleid(cost_nas)]
data[cost_nas==0,consecutive_nas_so_far:=0]
# total number of consecutive nas in the sequence
data[,total_number_of_consec_nas:=ifelse(consecutive_nas_so_far>0&shift(consecutive_nas_so_far,1,type = "lead")==0,consecutive_nas_so_far,NA)]
data[cost_nas==0,total_number_of_consec_nas:=0]
data[,total_number_of_consec_nas:=zoo::na.locf(total_number_of_consec_nas,fromLast=T)]
#get last and next known values for cost:
data[,cost_previous:=zoo::na.locf(cost)]
data[,cost_following:=zoo::na.locf(cost,fromLast=T)]
# apply the formula to calculate the gradual increase from cost_previous to cost_following
data[,cost_esti:=round(consecutive_nas_so_far*(cost_following-cost_previous)/(total_number_of_consec_nas+1)+cost_previous,0)]
data[is.na(cost_esti),cost_esti:=cost]
