1

I need to forward-fill nan values in a numpy array along the columns (axis=0). I am looking for a pure numpy solution that yields the same result as pd.fillna(method="ffill").

import numpy as np
import pandas as pd

arr = np.array(
    [
        [5, np.nan, 3],
        [4, np.nan, np.nan],
        [6, 2, np.nan],
        [2, np.nan, 6],
    ]
)

expected = pd.DataFrame(arr).fillna(method="ffill", axis=0)  # I need this line in pure numpy

print(f"Original array:\n {arr}\n")
print(f"Expected array:\n {expected.values}\n")

Original array:
 [[ 5. nan  3.]
 [ 4. nan nan]
 [ 6.  2. nan]
 [ 2. nan  6.]]

Expected array:
 [[ 5. nan  3.]
 [ 4. nan  3.]
 [ 6.  2.  3.]
 [ 2.  2.  6.]]
Andi
  • 3,196
  • 2
  • 24
  • 44
  • I had this same issue. Frustrated the hell out of me. Ended up using a chained pandas instruction. [pd.Series(array).bfill().to_numpy()]. – Paul Jul 30 '23 at 07:27

2 Answers2

2

No inbuilt function in numpy to do this. Below simple code will generate desired result using numpy array only.

row,col = arr.shape
mask = np.isnan(arr)
for i in range(1,row):
    for j in range(col):
        if mask[i][j]:
            arr[i][j] =arr[i-1][j]
Vikrant Gupta
  • 217
  • 2
  • 7
0

Bottleneck push function is a good option to forward fill. It's normally used internally in packages like Xarray.

from bottleneck import push
push(arr, axis=0)
Ashwiniku918
  • 281
  • 2
  • 7
  • Usually this answer would be sufficient, however, I am not able to install the package in my environment. Hence, I am looking for a pure ``numpy`` (or maybe ``numba``) solution. – Andi Jan 11 '22 at 11:45
  • This link might help with various : https://stackoverflow.com/questions/41190852/most-efficient-way-to-forward-fill-nan-values-in-numpy-array – Ashwiniku918 Jan 11 '22 at 11:47