Any changes in 'functools.lru_cache' from python 2 and python 3?

Question

My code is here works fine in python2.7 but fails inpython 3 functools.lru_cache(maxsize=32) any change in from python 2 to python 3.

The error I am getting is for my configparser object while caching in functools.lru_cacheit says

TypeError: unhashable type: 'ConfigParser'

Want to understand the changes in 'functools.lru_cache' from python 2 and python 3?

#CONFI FILE
[translate]
api_url = https://url
api_version = version_num
api_key = key_value


#code goes here
import functools
from configparser import ConfigParser as SafeConfigParser
config = SafeConfigParser()
path ="./conf/services.ini"
config.read(path)


@functools.lru_cache(maxsize=32)
def build_api_params_key(config):
    """Build the api url and return with key."""
    api_url = config.get('translate', 'api_url')
    api_version = config.get('translate', 'api_version')
    api_key = config.get('translate', 'api_key')
    full_api_url = api_url + api_version

    return api_key

`cache = {}` `cache_get = cache.get` .... `make_key = _make_key ` # build a key from the function `arguments` `key = make_key(args, kwds, typed)` `result = cache_get(key, sentinel)` The error is because the while caching the parser object is used as dict key, and since the object is not hashable it throws the error. Any other method to use functools.lru_cache if we have a unhashable object as argument? — , Nov 07 '19 at 13:09
[Python 2.7 does not have ``functools.lru_cache``.](https://docs.python.org/2.7/library/functools.html) It was added in Python 3.2. — MisterMiyagi, Nov 07 '19 at 13:52
Yes, Python 2.7 uses `functools32.lru_cache` I tried using `functools.lru_cache` since `functools32` is renamed as `functools` in python 3. — , Nov 07 '19 at 14:01
`functools32` was not renamed to `functools`. `functools32` is a *backport* of later `functools` functionality. — user2357112, Nov 08 '19 at 02:34

James Mchugh · Accepted Answer · 2019-11-08T02:22:07.830

The issue here is not functools.lru_cache, it is actually the ConfigParser. ConfigParser inherits from RawConfigParser, which in Python 3x, inherits from collections.abc.MutableMapping. The MutableMapping abstract class is not hashable, as it is mutable and does not implement the __hash__ magic method.

Since the ConfigParser instance is not hashable, it cannot be used as a key to the cache dictionary within the functools.lru_cache decorator.

For further reference, see this section of the configparser docs.

Assuming that it is necessary to cache the contents of the config file, another option would be to read the the contents of the file and then pass the contents string to the cached function, like so

import functools
from configparser import ConfigParser as SafeConfigParser
path = "./conf/services.ini"
config_contents = open(path).read()

@functools.lru_cache(maxsize=32)
def build_api_params_key(config_contents: str):
    """Build the api url and return with key."""
    config = SafeConfigParser()
    config.read_string(config_contents)
    api_url = config.get('translate', 'api_url')
    api_version = config.get('translate', 'api_version')
    api_key = config.get('translate', 'api_key')
    full_api_url = api_url + api_version

    return api_key

In the solution above, the config file is read to get string containing its contents. Since strings are hashable, this can be passed to the cached function. You can also do something similar with the file pointer if you would rather read the contents of the file within the function. However, these solutions are not compatible to Python 2.7 as read_string is not defined.

This is a great work around, indeed. But I have found something here, https://stackoverflow.com/questions/4669391/python-anyone-have-a-memoizing-decorator-that-can-handle-unhashable-arguments I liked the approach, thought of sharing it along with this. — , Nov 08 '19 at 09:39
@arunnaray Thank you for sharing this. This is a great solution to using a caching decorator to cache unhashable arguments. As your question was concerned with the difference between Python 2 and 3, would you say this answered you question? — James Mchugh, Nov 08 '19 at 11:51
@arunnaray If performance is a concern, I should also point out that in the case of your code in the question, using pickle to hash the arguments introduces a significant amount of overhead. In fact, it causes enough overhead that the `MemoizeMutable` decorated function is about 3x slower than the function without any caching. — James Mchugh, Nov 08 '19 at 13:05
The `configparser` PyPI package which backports many of the Python 3 changes also inherits from `MutableMapping`, yet remains hashable in Python 2. I suspect the story is slightly more complicated - maybe this ABC only began to be checked in Python 3? Since there was never an explicit `__hash__` it probably blocks the default `id` usage. — Sam Brightman, Oct 26 '20 at 17:19

Any changes in 'functools.lru_cache' from python 2 and python 3?

1 Answers1