dict.get(None) on a dict with None value takes default arg

Question

I'm trying to generalize this scenario:

def func(t):
    # code which doesn't accept None's

def foo(a):
    if a in ('', None):
        return None
    return func(a)

def foo_b(a):
    if a in ('', None):
        return None
    elif a == 'b'
         return 0
    return func(a)

The way I'm trying to solve it is by defining a dictionary:

defaults = {None: None, '': None}

def foo(a):
    return defaults.get(a, func(a))

def foo_b(a):
    defaults_b = {'bar': 0, **defaults}
    return defaults_b.get(a, func(a))

When running the script:

foo_b(None)

It gets to the get method, gets None back from defaults dict and goes inside func() method with None and I get errors cause func() method doesn't excepts None's

What is the best practice to solve this issue with python?

Answer 1

This can't work because all the arguments to a function are evaluated before the function is called. So when you write

defaults.get(a, func(a))

it first evaluates a and func(a), and passes both results to defaults.get(). Since func(a) doesn't work for some values of a, you get an error before ever getting into defaults.get().

The way to solve this in general is for the default to be a function, which will only be called when necessary. You'll need to write your own version of dict.get() that works this way.

def dict_get_f(d, key, default_fun):
    if key in d:
        return d[key]
    return default_fun()

You can then use it like this:

def foo(a):
    return dict_get_f(defaults, a, lambda: func(a))

Answer 2

Why not just check in the function itself?

def func(t):
    if t is None:
        return None
    # go on with the code

    

Answer 3

The Problem

As @Barmar pointed out in their answer, the problem is not the behaviour of dict.get, but the fact that all arguments are evaluated before the function is called, so defaults.get(a, func(a)) calls func(a) before it calls defaults.get using that value.

Solution 1: collections.defaultdict

An alternative way solving this is to use collections.defaultdict, which provides a way of calling a function when the value is not in the dictionary.

Since your dictionaries are small, copying them is low cost, so this solution does this.

from collections import defaultdict

defaults = {None: None, '': None}


def func(arg):
    """Some function that cannot be called with None"""
    if arg is None:
        raise ValueError()
    print(f'func called with {arg}')


def foo(a):
    return defaultdict(lambda: func(a), defaults)[a]


def foo_b(a):
    defaults_b = {'bar': 0, **defaults}
    return defaultdict(lambda: func(a), defaults_b)[a]

if __name__ == '__main__':
    print(foo(None))
    print(foo(''))
    print(foo(123))
    print(foo(123))
    print(foo_b(None))
    print(foo_b('bar'))

Solution 2: functions everywhere!

Another approach you could take would be to make the defaults dictionary a dictionary of functions, and use a default value of lambda: func(a), to avoid calling the function with None. So, defaults.get(a, lambda: func(a)) will return a function that returns the correct value, which you can then call to get the value you want: defaults.get(a, lambda: func(a))().

A full program:

defaults = {None: (lambda: None), '': (lambda: None)}

def func(arg):
    """Some function that cannot be called with None"""
    if arg is None:
        raise ValueError()
    print(f'func called with {arg}')


def foo(a):
    return defaults.get(a, lambda: func(a))()


def foo_b(a):
    defaults_b = {'bar': (lambda: 0), **defaults}
    return defaults_b.get(a, lambda: func(a))()


if __name__ == '__main__':
    print(foo(None))
    print(foo(''))
    print(foo(123))
    print(foo(123))
    print(foo_b(None))
    print(foo_b('bar'))

Solution 3: flashy and somewhat obfuscated boolean operation short-circuiting

As a third option, you could use the behaviour of and in python, replying upon the fact that all default values you list are "falsy" (What is Truthy and Falsy? How is it different from True and False?). In python, a and b is short-circuiting, meaning that if a is False or falsy, then it already knows that a and b cannot be True, so it returns a, or if a is truthy then it returns b.

So, if a is None, then defaults.get(a, True) and func(a) is equal to None and func(a). None is falsy, so None and func(a) is equal to None. If a is not in defaults then defaults.get(a, True) and func(a) is equal to True and func(a), which is equal to func(a), so the function gets called.

Note that this will not work if you use a truthy default value, e.g. if defaults = {None: None, '': None, 'bar': 'something'}, and it may be hard to understand for maintainability - I just thought I'd give another option.

defaults = {None: None, '': None}


def func(arg):
    """Some function that cannot be called with None"""
    if arg is None:
        raise ValueError()
    print(f'func called with {arg}')


def foo(a):
    return defaults.get(a, True) and func(a)


def foo_b(a):
    defaults_b = {'bar': 0, **defaults}
    return defaults_b.get(a, True) and func(a)