-1

I have a database which contains 2 tables. I'm reading both tables into a pandas dataframe (each one to its own) and than merge them into a single data frame. Both tables contains common value of time on which the data is than sorted and if there is some missing data, it is filled with previous known values. Due to the combining it can be that the values in the columns are either NaN or None.

df_A = pd.read_sql('Select * from A', db_connection)
df_B = pd.read_sql('Select * from B', db_connection)
df = pd.concat([df_A, df_B]).sort_values('Time')
df = df.fillna(method='pad')

Now on this dataframe I'm executing the apply function to perform some calculation on the data from both tables.

In the calculate function, I'd like to check whether the input data is containing some data (if one of the inputs is NaN or None, there is no sense to the calculation and there would be exception on it - which I'd like avoid before hand).

How can I check whether from_A (or from_B) is either either NaN or None? (or is there some other solution?)

df['new_data'] = df.apply(lambda x: calculate (x.A, x.B))

def calculate (from_A, from_B):
   do some calculations here
   return data

How can I check that the

user1977050
  • 496
  • 1
  • 6
  • 18
  • Hey, check this question out: https://stackoverflow.com/questions/944700/how-can-i-check-for-nan-values – ccommjin Aug 30 '21 at 14:26

1 Answers1

2

You can explicitly check this in the calculate function itself using the pd.isna() function :

import pandas as pd

def calculate (from_A, from_B):
   if (not pd.isna(from_A)) and (not pd.isna(from_B)):
       # perform the further calculation
   else:
      # do something else or skip
mabergerx
  • 1,216
  • 7
  • 19