My former co-worker has created some "manager" classes that are interfaces to key-value tables in our database. We have, for example, a ConfigurationManager, that maps to a Configuration table in the database that holds configuration values with keys (i.e.: "mailserver") and values (i.e.: "127.0.0.1").
This table might hold a few hundred of these key-value items, and we'd like to look one of them up occasionally from our code.
To prevent that every time we need a value from the configuration table, we need to do a query, he has implemented a method to "cache" all of the values in a list in the class, so it holds on to these lists in a class variable.
If the _loaded
variable is True
, he loads the list from the database, if not, he returns it from the class variable, like so:
class ConfigurationManager(object):
"""
Provides configuration option values
"""
_all_values = []
_loaded = False
@staticmethod
def value(name):
if not ConfigurationManager._loaded:
ConfigurationManager.load_database()
config = [x for x in ConfigurationManager._all_values if x.conf_name.lower() == name.lower()]
if len(config) > 0:
option = config[0]
return option.conf_value
else:
logger.debug('Configuration value {0} is empty or does not exist.'.format(name))
return None
@staticmethod
def load_database():
"""
Load configuration options from database
"""
from flask import current_app
ConfigurationManager._all_values = Configuration.query.all()
for confval in ConfigurationManager._all_values:
current_app.db_session.expunge(confval)
ConfigurationManager._loaded = True
Now, I must add that he is a C# developer originally, and this might be standard practice in C#. However, I have never seen an example like this in Python.
The actual problem is, that this seems to work when I test this in a unit test, but the code is run by a Huey Task worker, using a Flask instance in production. What we notice is that after some time the values can't be found any more, although they are present in the database. It seems like our class-variable-cache is emptied.
What I suspect is that either:
- Python garbage-collects these classes and thereby erases the class-variables at some point. However, this doesn't explain why the ConfigurationManager doesn't just refresh the list from the database.
- It has something to do with Flask and threads, where some objects are
used that are "loaded" and later on another instance is used that is
empty. However, the same thing goes here, the
_loaded
bit should indicate that the data needs to be loaded and it should, in theory, work.
If anyone can shed a light on if this method of "cacheing" works, and what could cause the variables to go missing in action, I would be most grateful.