Idiomatic way to inject parameters with python decorators

Question

So, this is a 2 part question -

1. Is there an idiomatic way in python to inject a parameter into the function signature when using a decorator?

For example:

def _mydecorator(func):
  def wrapped(someval, *args, **kwargs):
    do_something(someval)
    return func(*args, **kwargs)
  return wrapped

@_mydecorator
def foo(thisval, thatval=None):
  do_stuff()

The reason around this is when using SaltStack's runner modules You define funcs within the module, and you can call those functions via the 'salt-run' command. If the above example was a Salt runner module call 'bar', I could then run:

salt-run bar.foo aval bval

The salt-run imports the module and calls the function with the arguments you've given on the command line. Any function in the module that begins with a _ or that is in a class is ignored and cannot be run via salt-run.

So, I wanted to define something like a timeout decorator to control how long the function can run for.

I realize I could do something like:

@timeout(30)
def foo():
  ...

But I wanted to make that value configurable, so that I could run something like:

salt-run bar.foo 30 aval bval
salt-run bar.foo 60 aval bval

The above decorator works, but it feels like a hack, since it's changing the signature of the function and the user has no idea, unless they look at the decorator.

I have another case where I want to make a decorator for taking care of 'prechecks' before the functions in the Salt runner execute. However, the precheck needs a piece of information from the function it's decorating. Here's an example:

def _precheck(func):
  def wrapper(*args, **kwargs):
    ok = False
    if len(args) > 0:
      ok = run_prechecks(args[0])
    else:
      ok = run_prechecks(kwargs['record_id'])
    if ok:
      func(*args, **kwargs)
  return wrapper

@_precheck
def foo(record_id, this_val):
  do_stuff()

This also seems hackish, since it requires that the function that's being decorated, a) has a parameter called 'record_id' and that b) it's the first argument.

Now, because I'm writing all these functions, it's not a huge deal, but seems like there's probably a better way of doing this ( like not using decorators to try and solve this )

Answer 1

The way to dynamically define decorator arguments is not using the syntactic sugar (@). Like this: func = dec(dec_arguments)(func_name)(func_arguments)

import json
import sys

foo = lambda record_id, thatval=None: do_stuff(record_id, thatval)
def do_stuff(*args, **kwargs):
    # python3
    print(*args, json.dumps(kwargs))

def _mydecorator(timeout):
    print('Timeout: %s' % timeout)
    def decorator(func):
        def wrapped(*args, **kwargs):
            return func(*args, **kwargs)
        return wrapped
    return decorator


if __name__ == '__main__':
    _dec_default = 30
    l_argv = len(sys.argv)
    if  l_argv == 1:
        # no args sent
        sys.exit('Arguments missing')
    elif l_argv == 2:
        # assuming record_id is a required argument
        _dec_arg = _dec_default
        _args = 1
    else:
        # three or more args: filename 1 2 [...]
        # consider using optional keyword argument `timeout`
        # otherwise in combination with another optional argument it's a recipe for disaster
        # if only two arguments will be given - with current logic it will be tested for `timeoutedness`
        try:
            _dec_arg = int(sys.argv[1])
            _args = 2
        except (ValueError, IndexError):
            _dec_arg = _dec_default
            _args = 1
    foo = _mydecorator(_dec_arg)(foo)(*sys.argv[_args:])

Answer 2

There is no idiomatic way to do this in python 3.7 as far as I know. Indeed @functools.wraps only works when you do not modify the signature (see what does functools.wraps do ?)

However there is a way to do the same as @functools.wraps (exposing the full signature, preserving the __dict__ and docstring): @makefun.wraps. With this drop-in replacement for @wraps, you can edit the exposed signature.

from makefun import wraps

def _mydecorator(func):
  @wraps(func, prepend_args="someval")
  def wrapped(someval, *args, **kwargs):
    print("wrapper executes with %r" % someval)
    return func(*args, **kwargs)
  return wrapped

@_mydecorator
def foo(thisval, thatval=None):
  """A foo function"""
  print("foo executes with thisval=%r thatval=%r" % (thisval, thatval))

# let's check the signature
help(foo)

# let's test it
foo(5, 1)

Yields:

Help on function foo in module __main__:

foo(someval, thisval, thatval=None)
    A foo function

wrapper executes with 5
foo executes with thisval=1 thatval=None

You can see that the exposed signature contains the prepended argument.

The same mechanism applies for appending and removing arguments, as well as editing signatures more deeply. See makefun documentation for details (I'm the author by the way ;) )