5

This example is contrived but represents a real life situation:

  1. I have a python script that takes command line arguments.

  2. main() will parse the arguments, and pass them on to an intermediate function (caller_func in the code example)

  3. The intermediate function will then call a decorated function (fib() in the example) that is decorated with lru_cache from functools, and the maxsize of the cache is an argument to be accepted from command line and passed via the intermediate function.

How do I do this?

import argparse
from functools import lru_cache

def main():
    # boilerplate for parsing command line arguments
    parser = argparse.ArgumentParser()
    parser.add_argument("--cache_size", default="10")
    parser.add_argument("--fibo_num", default="20")
    args = parser.parse_args()
    cache_size = int(args.cache_size)
    fibo_num = int(args.fibo_num)

    caller_func(cache_size, fibo_num)

#Intermediate function that is supposed to call decorated function
def caller_func(cache_size, fib_num): 
    print(fib(fib_num))

#function decorated by LRU cache
@lru_cache(maxsize=cache_size)
def fib(n): 
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

if __name__ == "__main__":
    main()

run as

python3 example.py --cache_size 5 --fibo_num 30

throws 

NameError: name 'cache_size' is not defined

I tried making cache_size a global variable, but it didn't work, and I don't want globals anyway.

jonrsharpe
  • 115,751
  • 26
  • 228
  • 437
maverik
  • 774
  • 6
  • 17
  • 2
    The decorator is executed at definition time, before the call to main is even reached. – jonrsharpe Nov 26 '19 at 20:50
  • 1
    Perhaps use partial func as here: https://stackoverflow.com/a/25827070/3279716 – Alex Nov 26 '19 at 20:52
  • Nice diagnosis @jonrsharpe. What's the solution then? Is it possible to decorate this decorator from command line arguments? – maverik Nov 26 '19 at 20:53
  • You don't have to apply the decorator like that; given that the intermediate takes cache size anyway (what for up until now?) you can call `lru_cache(maxsize=cache_size)(fib)(fibo_num)`. – jonrsharpe Nov 26 '19 at 20:55
  • `lru_cache(maxsize=cache_size)(fib)(fibo_num)` won't do what you want, because it doesn't re-assign the function `fib` to the decorated version; so the recursive calls will be on the undecorated `fib` function which doesn't use the cache. – kaya3 Nov 26 '19 at 21:25

1 Answers1

5

You don't have to use a decorator with decorator syntax. You can wait to "decorate" fib until after you have the desired cache size. For example,

import argparse
from functools import lru_cache

def main():
    global fib
    # boilerplate for parsing command line arguments
    parser = argparse.ArgumentParser()
    parser.add_argument("--cache_size", default="10")
    parser.add_argument("--fibo_num", default="20")
    args = parser.parse_args()
    cache_size = int(args.cache_size)
    fibo_num = int(args.fibo_num)

    fib = lru_cache(maxsize=cache_size)(fib)

    caller_func(fibo_num)

#Intermediate function that is supposed to call decorated function
def caller_func(fib_num): 
    print(fib(fib_num))

def fib(n): 
    if n < 2:
        return n
    return fib(n-1) + fib(n-2)

if __name__ == "__main__":
    main()
chepner
  • 497,756
  • 71
  • 530
  • 681