31

I have a function foo that calls math.isclose:

import math
def foo(..., rtol=None, atol=None):
    ...
    if math.isclose(x, y, rel_tol=rtol, abs_tol=atol):
        ...
    ...

The above fails in math.isclose if I do not pass rtol and atol to foo:

TypeError: must be real number, not NoneType

I do not want to put the system default argument values in my code (what if they change down the road?)

Here is what I came up with so far:

import math
def foo(..., rtol=None, atol=None):
    ...
    tols = {}
    if rtol is not None:
        tols["rel_tol"] = rtol
    if atol is not None:
        tols["abs_tol"] = atol
    if math.isclose(x, y, **tols):
        ...
    ...

This looks long and silly and allocates a dict on each invocation of foo (which calls itself recursively so this is a big deal).

So, what is the best way to tell math.isclose to use the default tolerances?

PS. There are several related questions. Note that I do not want to know the actual default arguments of math.isclose - all I want it to tell it to use the defaults whatever they are.

sds
  • 58,617
  • 29
  • 161
  • 278
  • 1
    you can grab the values of the default args: [Get a function argument's default value?](https://stackoverflow.com/questions/12627118/get-a-function-arguments-default-value) and come up with a way to define your function using those values. There's probably a way using `functools.partial` – pault Apr 03 '19 at 14:46
  • possible duplicate of https://stackoverflow.com/questions/4670665/conditionally-passing-arbitrary-number-of-default-named-arguments-to-a-function but you will not find better answer in it. – naivepredictor Apr 03 '19 at 14:49
  • 2
    @pault I tried `inspect.signature(math.isclose)` and it fails with `ValueError: no signature found for builtin `. – sanyassh Apr 03 '19 at 14:50
  • @Sanyash It works for me in python 3.7.2. – Aran-Fey Apr 03 '19 at 14:53
  • @Aran-Fey indeed, in 3.7.2 it works, but not in 3.6.6 – sanyassh Apr 03 '19 at 14:54
  • Could you use an if else? For example: `if rtol and math.isclose(x, y, rel_tol=rtol, abs_tol=atol):` `elif math.isclose(x, y):` – Error - Syntactical Remorse Apr 03 '19 at 15:00
  • Define a recursive helper function that `foo` calls so you only have to define the dict once. – chepner Apr 03 '19 at 15:00
  • @Error-SyntacticalRemorse: this is _precisely_ what I am trying to avoid, the a deleted answer. – sds Apr 03 '19 at 15:05
  • @sds You just want a single if statement, not two of them? – Error - Syntactical Remorse Apr 03 '19 at 15:06
  • Possible duplicate of [Using default arguments in a function with variable arguments. Is this possible?](https://stackoverflow.com/questions/53967210/using-default-arguments-in-a-function-with-variable-arguments-is-this-possible) – Silvio Mayolo Apr 03 '19 at 15:31
  • If MartijnPieters' answer doesn't work for you, I would honestly just do what you already have. – jpmc26 Apr 03 '19 at 22:47

4 Answers4

12

One way to do it would be with variadic argument unpacking:

def foo(..., **kwargs):
    ...
    if math.isclose(x, y, **kwargs):
        ...

This would allow you to specify atol and rtol as keyword arguments to the main function foo, which it would then pass on unchanged to math.isclose.

However, I would also say that it is idiomatic that arguments passed to kwargs modify the behaviour of a function in some way other than to be merely passed to sub-functions being called. Therefore, I would suggest that instead, a parameter is named such that it is clear that it will be unpacked and passed unchanged to a sub-function:

def foo(..., isclose_kwargs={}):
    ...
    if math.isclose(x, y, **isclose_kwargs):
        ...

You can see an equivalent pattern in matplotlib (example: plt.subplots - subplot_kw and gridspec_kw, with all other keyword arguments being passed to the Figure constructor as **fig_kw) and seaborn (example: FacetGrid - subplot_kws, gridspec_kws).

This is particularly apparent when there are mutiple sub-functions you might want to pass keyword arguments, but retain the default behaviour otherwise:

def foo(..., f1_kwargs={}, f2_kwargs={}, f3_kwargs={}):
    ...
    f1(**f1_kwargs)
    ...
    f2(**f2_kwargs)
    ...
    f3(**f3_kwargs)
    ...

Caveat:

Note that default arguments are only instantiated once, so you should not modify the empty dicts in your function. If there is a need to, you should instead use None as the default argument and instantiate a new empty dict each time the function is run:

def foo(..., isclose_kwargs=None):
    if isclose_kwargs is None:
        isclose_kwargs = {}
    ...
    if math.isclose(x, y, **isclose_kwargs):
        ...

My preference is to avoid this where you know what you're doing since it is more brief, and in general I don't like rebinding variables. However, it is definitely a valid idiom, and it can be safer.

gmds
  • 19,325
  • 4
  • 32
  • 58
  • Further, *document* that any additional keyword arguments to `foo` are passed directly to `math.isclose`. – chepner Apr 03 '19 at 14:58
  • @chepner Yup, that's how the `FacetGrid` example does it. In addition, I think that in a case where keyword arguments are being passed on, an explicit `isclose_kwargs` or similar argument would be preferred to naked `**kwargs`. – gmds Apr 03 '19 at 15:01
  • Don't use mutable default arguments, however. `f1_kwargs=None` is better. – Martijn Pieters Apr 03 '19 at 15:10
  • @MartijnPieters if you don't mutate it in your function itself, it makes your code less verbose to use `{}` as a default argument. – gmds Apr 03 '19 at 15:19
  • 1
    @gmds: I can still use `foo.__defaults__[0]['bar'] = 42` to mess with the defaults. And future refactorings of the function could easily miss that you defined these as defaults. Don't make it that easy to introduce errors. – Martijn Pieters Apr 03 '19 at 15:25
  • @gmds: and don't underestimate the number of people that'll copy and paste from Stack Overflow and then do modify the dictionaries. Better to use best-practices *in your answers*. – Martijn Pieters Apr 03 '19 at 15:27
  • 1
    @MartijnPieters Anyone with the knowledge to modify default arguments in that way is good enough with Python to know that default arguments are instantiated only once. As for the rest, again, a sufficiently skilled programmer will know when to break the rules for a good reason - in this case, brevity. I have edited my answer to note the default argument pitfall, but I think that is sufficient. – gmds Apr 03 '19 at 15:33
11

The correct solution would be to use the same defaults as math.isclose(). There is no need to hard-code them, as you can get the current defaults with the inspect.signature() function:

import inspect
import math

_isclose_params = inspect.signature(math.isclose).parameters

def foo(..., rtol=_isclose_params['rel_tol'].default, atol=_isclose_params['abs_tol'].default):
    # ...

Quick demo:

>>> import inspect
>>> import math
>>> params = inspect.signature(math.isclose).parameters
>>> params['rel_tol'].default
1e-09
>>> params['abs_tol'].default
0.0

This works because math.isclose() defines its arguments using the Argument Clinic tool:

[T]he original motivation for Argument Clinic was to provide introspection “signatures” for CPython builtins. It used to be, the introspection query functions would throw an exception if you passed in a builtin. With Argument Clinic, that’s a thing of the past!

Under the hood, the math.isclose() signature is actually stored as a string:

>>> math.isclose.__text_signature__
'($module, /, a, b, *, rel_tol=1e-09, abs_tol=0.0)'

This is parsed out by the inspect signature support to give you the actual values.

Not all C-defined functions use Argument Clinic yet, the codebase is being converted on a case-by-case basis. math.isclose() was converted for Python 3.7.0.

You could use the __doc__ string as a fallback, as in earlier versions this too contains the signature:

>>> import math
>>> import sys
>>> sys.version_info
sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
>>> math.isclose.__doc__.splitlines()[0]
'isclose(a, b, *, rel_tol=1e-09, abs_tol=0.0) -> bool'

so a slightly more generic fallback could be:

import inspect

def func_defaults(f):
    try:
        params = inspect.signature(f).parameters
    except ValueError:
        # parse out the signature from the docstring
        doc = f.__doc__
        first = doc and doc.splitlines()[0]
        if first is None or f.__name__ not in first or '(' not in first:
            return {}
        sig = inspect._signature_fromstr(inspect.Signature, math.isclose, first)
        params = sig.parameters
    return {
        name: p.default for name, p in params.items()
        if p.default is not inspect.Parameter.empty
    }

I'd see this as a stop-gap measure only needed to support older Python 3.x releases. The function produces a dictionary keyed on parameter name:

>>> import sys
>>> import math
>>> sys.version_info
sys.version_info(major=3, minor=6, micro=8, releaselevel='final', serial=0)
>>> func_defaults(math.isclose)
{'rel_tol': 1e-09, 'abs_tol': 0.0}

Note that copying the Python defaults is very low risk; unless there is a bug, the values are not prone to change. So another option could be to hardcode the 3.5 / 3.6 known defaults as a fallback, and use the signature provided in 3.7 and newer:

try:
    # Get defaults through introspection in newer releases
    _isclose_params = inspect.signature(math.isclose).parameters
    _isclose_rel_tol = _isclose_params['rel_tol'].default
    _isclose_abs_tol = _isclose_params['abs_tol'].default
except ValueError:
    # Python 3.5 / 3.6 known defaults
    _isclose_rel_tol = 1e-09
    _isclose_abs_tol = 0.0

Note however that you are at greater risk of not supporting future, additional parameters and defaults. At least the inspect.signature() approach would let you add an assertion about the number of parameters that your code expects there to be.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
7

There really aren't many ways to make a function use its default arguments... You only have two options:

  1. Pass the real default values
  2. Don't pass the arguments at all

Since none of the options are great, I'm going to make an exhaustive list so you can compare them all.

  • Use **kwargs to pass through arguments

    Define your method using **kwargs and pass those to math.isclose:

    def foo(..., **kwargs):
    ...
    if math.isclose(x, y, **kwargs):

    Cons:

    • the parameter names of both functions have to match (e.g. foo(1, 2, rtol=3) won't work)

  • Manually construct a **kwargs dict

    def foo(..., rtol=None, atol=None):
    ...
    kwargs = {}
    if rtol is not None:
        kwargs["rel_tol"] = rtol
    if atol is not None:
        kwargs["abs_tol"] = atol
    if math.isclose(x, y, **kwargs):

    Cons:

    • ugly, a pain to code, and not fast

  • Hard-code the default values

    def foo(..., rtol=1e-09, atol=0.0):
    ...
    if math.isclose(x, y, rel_tol=rtol, abs_tol=atol):

    Cons:

    • hard-coded values

  • Use introspection to find the default values

    You can use the inspect module to determine the default values at run time:

    import inspect, math

signature = inspect.signature(math.isclose)
DEFAULT_RTOL = signature.parameters['rel_tol'].default
DEFAULT_ATOL = signature.parameters['abs_tol'].default

def foo(..., rtol=DEFAULT_RTOL, atol=DEFAULT_ATOL):
    ...
    if math.isclose(x, y, rel_tol=rtol, abs_tol=atol):

    Cons:

Aran-Fey
  • 39,665
  • 11
  • 104
  • 149
3

Delegate the recursion to a helper function so that you only need to build the dict once.

import math
def foo_helper(..., **kwargs):
    ...
    if math.isclose(x, y, **kwargs):
        ...
    ...


def foo(..., rtol=None, atol=None):
    tols = {}
    if rtol is not None:
        tols["rel_tol"] = rtol
    if atol is not None:
        tols["abs_tol"] = atol

    return foo_helper(x, y, **tols)

Or, instead of passing tolerances to foo, pass a function which already incorporates the desired tolerances.

from functools import partial

# f can take a default value if there is a common set of tolerances
# to use.
def foo(x, y, f):
    ...
    if f(x,y):
        ...
    ...

# Use the defaults
foo(x, y, f=math.isclose)
# Use some other tolerances
foo(x, y, f=partial(math.isclose, rtol=my_rtel))
foo(x, y, f=partial(math.isclose, atol=my_atol))
foo(x, y, f=partial(math.isclose, rtol=my_rtel, atol=my_atol))
chepner
  • 497,756
  • 71
  • 530
  • 681