Can fillna take in a function? Or just the methods ‘backfill’, ‘bfill’, ect ?
http://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.fillna.html
If the dataframe does not have any initial data for example, the backfill method would be useless isn't it? viceversa ffill in case no data at end of dataframe.
That would be some of the reasons why I would like to use a proper method like extrapolations
fillna is intended to take either a value to fill with, or use one of the methods *{‘backfill’, ‘bfill’, ‘pad’, ‘ffill’, None}*
value : scalar, dict, Series, or DataFrame
Value to use to fill holes (e.g. 0), alternately a dict/Series/DataFrame of values specifying which value to use for each index (for a Series) or column (for a DataFrame). (values not in the dict/Series/DataFrame will not be filled). This value cannot be a list.
Answer
fillna does not take a function as a valid input for the value parameter. However, if you have a function that can produce a series or dataframe, then you can pass what ever inputs you need into the function and pass the results to fillna
examples
the value parameter can take scalars, dictionaries, series, or dataframes. Here are some examples of what that looks like
Consider the dataframe df
df = pd.DataFrame([
[1, None, 2, None],
[None, 3, None, 4],
[5, 6, None, None],
[None, None, 7, 8]
], list('ABCD'), list('WXYZ'))
print(df)
W X Y Z
A 1.0 NaN 2.0 NaN
B NaN 3.0 NaN 4.0
C 5.0 6.0 NaN NaN
D NaN NaN 7.0 8.0
fill with scalar
df.fillna(9)
W X Y Z
A 1.0 9.0 2.0 9.0
B 9.0 3.0 9.0 4.0
C 5.0 6.0 9.0 9.0
D 9.0 9.0 7.0 8.0
fill with dictionary
filler = dict(W=-9, X=-10, Y=-11, Z=-12)
df.fillna(filler)
W X Y Z
A 1.0 -10.0 2.0 -12.0
B -9.0 3.0 -11.0 4.0
C 5.0 6.0 -11.0 -12.0
D -9.0 -10.0 7.0 8.0
fill with series
filler = pd.Series(dict(W=-9, X=-10, Y=-11, Z=-12))
df.fillna(filler)
W X Y Z
A 1.0 -10.0 2.0 -12.0
B -9.0 3.0 -11.0 4.0
C 5.0 6.0 -11.0 -12.0
D -9.0 -10.0 7.0 8.0
fill with dataframe
filler = pd.DataFrame(-np.arange(df.size).reshape(df.shape), df.index, df.columns)
print(filler)
W X Y Z
A 0 -1 -2 -3
B -4 -5 -6 -7
C -8 -9 -10 -11
D -12 -13 -14 -15
df.fillna(filler)
W X Y Z
A 1.0 -10.0 2.0 -12.0
B -9.0 3.0 -11.0 4.0
C 5.0 6.0 -11.0 -12.0
D -9.0 -10.0 7.0 8.0
fill with along index, you need to transpose
df.fillna(filler, axis=1) generates a NotImplementedError
filler = dict(A=-9, B=-10, C=-11, D=-12)
print(df.T.fillna(filler).T)
W X Y Z
A 1.0 -9.0 2.0 -9.0
B -10.0 3.0 -10.0 4.0
C 5.0 6.0 -11.0 -11.0
D -12.0 -12.0 7.0 8.0
Short answer:
No, fillna cannot take a function but only the methods.
Less Short answer:
There is a df.interpolate function that allows you to fillna using different interpolation technique.
Long answer:
You can refer to the answer here, where someone has outlined a way to use scipy to extrapolate data. Unfortunately there isn't a native pandas function to accomplish this ):
