Counting NaN values in a dataframe

Question

I try to analyse wind speed data from a lidar, creating a dataframe in which the columns are the investigated heights and the row is the number of NaNs at that elevation. My script creates the dataframe and names the columns as it required but it doesn't write the number of NaNs in the corresponding cells. Any idea what the problem might be?

df=pd.read_csv(fileApath,delimiter=',',skiprows=1)
heights = ['123','98','68','65','57','48','39','38','29','18','10']
nanvalues_speed=pd.DataFrame()
for i in heights:
    nanvalues_speed[i+'m']=pd.notnull(df['Horizontal Wind Speed (m/s) at '+i+'m']).sum()

I think this would help you : https://stackoverflow.com/questions/26266362/how-to-count-the-nan-values-in-a-column-in-pandas-dataframe — Chitrank Dixit, Dec 02 '19 at 09:48
Can you add some data sample, 3-4 rows, 2-3 columns with expected output? — jezrael, Dec 02 '19 at 09:50
Maybe you meant to use `pd.isna()` instead of `pd.notnull()`? — Aryerez, Dec 02 '19 at 10:45

score 0 · Accepted Answer · answered Dec 02 '19 at 10:54

0

The function you are looking for is pandas.DataFrame.isna()
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.isna.html

import pandas as pd
import numpy as np

df = pd.DataFrame({'a': [1,2,3,np.nan ,5], 
                   'b': ['a',np.nan ,'c','d','e']})

df.isna().sum()

The pandas.DataFrame.isna() funtion returns a boolean same-sized object indicating if the values in the DataFrame are NA.

answered Dec 02 '19 at 10:54

julian

451
2
8

thanks. isna and sum don't work if they are in the same line but they are ok if separated – blooregard Dec 02 '19 at 11:30

Counting NaN values in a dataframe

1 Answers1