Analogue of devar in Python

Question

When writing Python code, I often find myself wanting to get behavior similar to Lisp's defvar. Basically, if some variable doesn't exist, I want to create it and assign a particular value to it. Otherwise, I don't want to do anything, and in particular, I don't want to override the variable's current value.

I looked around online and found this suggestion:

try:
    some_variable
except NameError:
    some_variable = some_expensive_computation()

I've been using it and it works fine. However, to me this has the look of code that's not paradigmatically correct. The code is four lines, instead of the 1 that would be required in Lisp, and it requires exception handling to deal with something that's not "exceptional."

The context is that I'm doing interactively development. I'm executing my Python code file frequently, as I improve it, and I don't want to run some_expensive_computation() each time I do so. I could arrange to run some_expensive_computation() by hand every time I start a new Python interpreter, but I'd rather do something automated, particularly so that my code can be run non-interactively. How would a season Python programmer achieve this?

I'm using WinXP with SP3, Python 2.7.5 via Anaconda 1.6.2 (32-bit), and running inside Spyder.

PINL (Python Is Not LISP)... exceptions are commonly used for flow control in Python (see e.g. `StopIteration`). — kindall, Oct 06 '13 at 22:49
if you're doing this within the context of a class you can do `if hasattr(self, 'some_variable')` but honestly using exceptions for flow control is fine, as @kindall says. — roippi, Oct 06 '13 at 22:55
For posterity: anyone who is using Spyder and saving variables while executing a Python file repeatedly should be careful of [Spyder's UMD](http://stackoverflow.com/questions/13876306/spyder-umd-has-deleted-module). Running "import bs4" broke all existing bs4.BeautifulSoup objects (giving one of several errors whose ultimate cause took some doing to find). The solution was to visit Tools -> Preferences -> Console -> Advanced Settings and add bs4 to the list of excluded modules. — kuzzooroo, Oct 07 '13 at 12:09
I think you accidentally a letter in the title of your question — porglezomp, Aug 26 '16 at 01:13

score 2 · Answer 1 · answered Oct 06 '13 at 23:00

It's generally a bad idea to rely on the existence or not of a variable having meaning. Instead, use a sentinel value to indicate that a variable is not set to an appropriate value. None is a common choice for this kind of sentinel, though it may not be appropriate if that is a possible output of your expensive computation.

So, rather than your current code, do something like this:

# early on in the program
some_variable = None

# later:
if some_variable is None:
    some_variable = some_expensive_computation()

# use some_variable here

Or, a version where None could be a significant value:

_sentinel = object()
some_variable = _sentinel # this means it doesn't have a meaningful value

# later
if some_variable is _sentinel:
    some_variable = some_expensive_computation()

The issue that I'm struggling with is that without doing something clever, "some_variable = None" will get run every time I execute the buffer. — kuzzooroo, Oct 06 '13 at 23:06
I'm not sure there is a good way to handle a persistent environment when you want to be able to rerun your code as you edit it. Maybe you could do something with `reload` (`imp.reload` in Python 3), but it would be very hackish. A possible alternative might be to save your expensive-to-calculate data somewhere outside the running code, like in a pickle file. — Blckknght, Oct 06 '13 at 23:35

score 2 · Answer 2 · answered Oct 06 '13 at 23:10

It is hard to tell which is of greater concern to you, specific language features or a persistent session. Since you say:

The context is that I'm doing interactively development. I'm executing my Python code file frequently, as I improve it, and I don't want to run some_expensive_computation() each time I do so.

You may find that IPython provides a persistent, interactive environment that is pleasing to you.

score 1 · Answer 3 · answered Oct 06 '13 at 23:00

1

Instead of writing Lisp in Python, just think about what you're trying to do. You want to avoid calling an expensive function twice and having it run two times. You can write your function do to that:

def f(x):
    if x in cache:
        return cache[x]

    result = ...
    cache[x] = result

    return result

Or make use of Python's decorators and just decorate the function with another function that takes care of the caching for you. Python 3.3 comes with functools.lru_cache, which does just that:

import functools

@functools.lru_cache()
def f(x):
    return ...

There are quite a few memoization libraries in the PyPi for 2.7.

answered Oct 06 '13 at 23:00

Blender

289,723
53
439
496

I think the question here is how to create the cache variable without resetting it every time I execute my file. I can't have a line that says "cache = {}" unless it's somehow guarded from running if the variable already exists. I take your point about memoization, though. Some or all of these libraries likely have built-in smarts to keep them from resetting their caches. – kuzzooroo Oct 07 '13 at 13:55
@kuzzooroo: I never said that `cache` was a dictionary. You could use the `shelve` module and store everything on disk. – Blender Oct 07 '13 at 20:35
Using shelve (designed for persistence) is a neat suggestion. Do you know if it's safe to open the same file as a shelf over and over again (every time I execute my code)? In trying this I believe I have several shelves of the same file going at the same time, and calling gc.collect doesn't seem to necessarily make them go away. – kuzzooroo Oct 08 '13 at 12:02
Looks like there's a backport of functools for Python 2.7: https://pypi.python.org/pypi/functools32 – kuzzooroo Oct 19 '13 at 15:21

alexis · Answer 4 · 2013-10-11T09:15:46.003

For the use case you give, guarding with a try ... except seems like a good way to go about it: Your code is depending on leftover variables from a previous execution of your script.

But I agree that it's not a nice implementation of the concept "here's a default value, use it unless the variable is already set". Python does not directly support this for variables, but it does have a default-setter for dictionary keys:

myvalues = dict()
myvalues.setdefault("some_variable", 42)
print some_variable    # prints 42

The first argument of setdefault must be a string containing the name of the variable to be defined.

If you had a complicated system of settings and defaults (like emacs does), you'd probably keep the system settings in their own dictionary, so this is all you need. In your case, you could also use setdefault directly on global variables (only), with the help of the built-in function globals() which returns a modifiable dictionary:

globals().setdefault("some_variable", 42)

But I would recommend using a dictionary for your persistent variables (you can use the try... except method to create it conditionally). It keeps things clean and it seems more... pythonic somehow.

"A more realistic solution would be to use a dictionary for your key store, instead of the global namespace; then `defvar` would check there and act accordingly." How would you suggest I avoid resetting the dictionary each time I execute my code file? The `try ... except` technique? — kuzzooroo, Oct 07 '13 at 13:58
Yeah, I'd do that for the dictionary itself (so just once); then you can use the standard `dict` method `setdefault`, which does exactly what you want (and also returns the correct value). If you're willing to use a `dict`, this is definitely the way to go.... hmm, I just realized you can use `setdefault` on `globals()` too! — alexis, Oct 07 '13 at 14:30

score 0 · Answer 5 · answered Oct 08 '13 at 03:15

Let me try to summarize what I've learned here:

Using exception handling for flow control is fine in Python. I could do it once to set up a dict in which I can store what ever I want.
There are libraries and language features that are designed for some form of persistence; these can provide "high road" solutions for some applications. The shelve module is an obvious candidate here, but I would construe "some form of persistence" broadly enough to include @Blender's suggest to use memoization.

Analogue of devar in Python

5 Answers5