Negation in np.select() condition

Question

Here is my code:

import pandas as pd
import numpy as np

df = pd.DataFrame({ 'var1': ['a', 'b', 'c',np.nan, np.nan],
                   'var2': [1, 2, np.nan , 4, np.nan]
                 })



conditions = [
    (not(pd.isna(df["var1"]))) & (not(pd.isna(df["var2"]))),
    (pd.isna(df["var1"])) & (pd.isna(df["var2"]))]

choices = ["No missing", "Both missing"]

df['Result'] = np.select(conditions, choices, default=np.nan)

Output:

  File "C:\ProgramData\Anaconda3\lib\site-packages\pandas\core\generic.py", line 1478, in __nonzero__
    f"The truth value of a {type(self).__name__} is ambiguous. "

ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Problem is with line (not(pd.isna(df["var1"]))) & (not(pd.isna(df["var2"]))). This line should give TRUE when in both var1 and var2 in not a NaN value. Problem here is with negation, because with conditions without negation there is no problem.

Question: How can to correct (not(pd.isna(df["var1"]))) & (not(pd.isna(df["var2"]))) line so in case when in both var1 and var2 in not a NaN value the condition should give TRUE?

why not use `df.notna()` instead: `conditions = [ (df["var1"].notna() & df['var2'].notna()), (pd.isna(df["var1"])) & (pd.isna(df["var2"]))]` — anky, Feb 06 '20 at 16:07
@anky_91 That worked, thank you. That much better way for this particular case. But I am also interested how to properly use negation for future use. What is wrong with my negation? — vasili111, Feb 06 '20 at 16:10
you can read [this](https://stackoverflow.com/a/44345755/9840637) answer for better explanation on why your negation doesn't work — anky, Feb 06 '20 at 16:19
https://stackoverflow.com/questions/36921951/truth-value-of-a-series-is-ambiguous-use-a-empty-a-bool-a-item-a-any-o — AMC, Feb 08 '20 at 22:01

Simon Rogers · Accepted Answer · 2020-02-06T17:28:25.473

1

Try:

conditions = [(~pd.isna(df["var1"]) & ~pd.isna(df["var2"])),
               (pd.isna(df["var1"]) &  pd.isna(df["var2"]))]

edited Feb 06 '20 at 17:28

answered Feb 06 '20 at 16:10

Simon Rogers

344
1
10

That worked. Thank you. what is the difference between with `.values` and without it? Which way is better and why? – vasili111 Feb 06 '20 at 16:14
Also can you please provide any link with `~` operator explanation (documentation)? – vasili111 Feb 06 '20 at 16:27
Thank you. What is the advantage of using array instead of Series in this case? – vasili111 Feb 06 '20 at 16:34
'.values' converts it from a Series to an array (not required). For more about "~" see here: wiki.python.org/moin/BitwiseOperators – Simon Rogers Feb 06 '20 at 17:29

Negation in np.select() condition

1 Answers1

Linked