Django: Python global variables overlap, even for separate runs

Question

I have a Django (so Python) program with a global variable:

g_variable = []

I use this is several functions where I also change the value:

my_function()
    global g_variable 
    g_variable.append(some_value)

That worked great until I started calling the program multiple overlapping times - in Django that means that I loaded the webpage multiple times quickly. I expected that the global variable would only be global within each individual run, but that is not the case. The values that are appended to g_variable in one run can be seen in the next run.

To me this means that I now have to pass this variable around to all my functions:

my_function(non_g_variable)
    non_g_variable.append(some_value)
    return non_g_variable

called with

non_g_variable = my_function(non_g_variable)

Is that correct? Before I change all my code I just want to make sure that I haven't missed something. It will add a lot of extra lines and return calls.

What is this variable doing? There's almost certainly a better way of doing it. — Daniel Roseman, May 21 '12 at 11:56
It's an object that represents a webpage, essentially all the elements and attributes, parent-child relationships, etc. This is used throughout my code. — user984003, May 21 '12 at 12:04
I'm going to need to see a real example. It seems like you are building up your page in a very unusual and non-Djangonic way. Usually you'd just call a single view function to build up the context, and pass it to a template for rendering. — Daniel Roseman, May 21 '12 at 12:24
It's not a Django thing - It's not the html for the template that I am building. It's a webpage that I am given, and which I then have to parse. Once it is parsed I then have to access it in various ways. I therefore have helper functions like is_it_am_image() or does_it_contain_a_link(). These functions access the global variable which essentially contains the webpage structure. I guess I could do it like a new class. Seems like it would involve as much rewritting, but maybe is more proper. — user984003, May 21 '12 at 12:32

score 3 · Answer 1 · edited May 23 '17 at 12:04

You should probably redesign your code to get rid of the global variable, as other answers and comments say. Something along the lines of:

class WebpageStructure(object):
    def __init__(self, html):
         # parse the html
         self.structure = self.parse(html)
    def contains_link(self):
         # figure it out using self.structure
         return ...

# in the view(s)
webpage = WebpageStructure(html_to_parse)
if webpage.contains_link():
    ...

There are however options:

If your code always runs in a single thread you can fix the problem by setting g_variable to [] between each run. There is probably one top-level function (a Django view function perhaps?) that always marks the start of each run. You should re-initialize the g_variable in this top-level function.
If your code runs multi-threaded, you cannot use a normal global variable. Concurrent threads will update the same global variable.

Regarding 1 and 2: To run a Django site in a single thread, use manage.py runserver --nothreading for the development server. If you host your site in apache/mod_wsgi, you can control this using daemon mode. Note that you can run multiple single-threaded side-by-side processes. Using a global variable will work in that scenario, since the processes are isolated.

If possible your code should work in any process/thread model.
If your code runs multi-threaded and you really want to avoid passing around the g_variable list you can use thread-local variables. Documentation here and here.

Example:

import threading
threadlocal = threading.local()

def mydjangoview(request):
    # In your top-level view function, initialize the list
    threadlocal.g_variable = []
    # Then call the functions that use g_variable
    foo()
    bar()

    # ... and then I guess you probably return a response?
    return Response(...)

def foo():
    threadlocal.g_variable.append(some_value)

def bar():
    threadlocal.g_variable.append(some_other_value)

Other links:

Always +1 when someone tries to explain threading and locals. — Burhan Khalid, May 21 '12 at 12:44

score 2 · Answer 2 · answered May 21 '12 at 12:31

That's how global variables work in Python. The global state persists for as long as the web application server keeps running.

A common solution would be to put your functions in a class, store the per-request state on that class and use a new instance of that class for each request.

score 1 · Answer 3 · answered May 21 '12 at 12:23

Using global variables, except for constants, is almost always error-prone, and often produces code that is difficult to read. Therefore, it is the wrong concept. Receiving an argument, modifying and returning it is much more explicit about what a function is doing than on-the-fly modification of a global variable.

So yes, I would go ahead and implement the receive-modify-return concept or wait until someone has a special "djangonic" solution to your problem.

score 0 · Answer 4 · answered May 21 '12 at 12:43

0

Your problems can be resolved by implementing a cache - a simplified explanation of a cache is - a global variable across a session.

answered May 21 '12 at 12:43

Burhan Khalid

169,990
18
245
284

Django: Python global variables overlap, even for separate runs

4 Answers4