2

I'm trying to integrate Flask with Dill to dump/load Python sessions on the server side. The code below has two functions, the first one sets the value of x to zero and imports the datetime library. The second one increments x by 1 and gets the timestamp.

The first function dumps the session and the second function loads it.

In the dump, the pickle file is generated correctly, but I cannot reuse x or get the timestamp.

This is the error when I try to execute x = x + 1 in the second function:

UnboundLocalError: local variable 'x' referenced before assignment

Can Dill be used with Flask? Do I need a different approach?

The code:

from flask import Flask
from dill import dump_session, load_session

app = Flask(__name__)
app.config['SECRET_KEY'] = 'super secret'

session_file = '/tmp/session.pkl'

@app.route('/start_counter')
def start_counter():
    import datetime
    x = 0
    dump_session(filename = session_file)
    return 'New counter started!'

@app.route('/count')
def count():
    load_session(filename = session_file)
    x = x + 1
    now = datetime.datetime.now()
    dump_session(filename = session_file)
    return str(x) + '-' + str(now)
ItsMe
  • 395
  • 2
  • 13
ps0604
  • 1,227
  • 23
  • 133
  • 330
  • You can do it manually too, right click on confusion_matrix image and save it. – Abdullah Oct 30 '21 at 17:27
  • @ps0604 within function local variables cannot be updated because of how functions memory are handled differently [check](https://stackoverflow.com/a/8028772/14475852) here for more detail. – Chandan Nov 05 '21 at 04:32

1 Answers1

3

How to fix?

To make things simple, you need a data structure to hold your application state. I would use a dict because it's simple, but you can define a class for that too.

The easy (and tedious) way is to call state = dill.load('filename')and dill.dump(object,'filename') every time you need your application state.

This will work if your application is small. If you need to maintain a proper application state, you should use a database.

Ok. But WHAT happened here?

There are no compatibility issues with dill and Flask.

When you call dill.dump_session() it saves the state of __main__.

But, when you increase x in function count(), it is undefined because it wasn't saved by dill.

An easy way to see that is to put a breakpoint() before x = x + 1 or print the content inside a try..except clause:

try:
   print(x)
except ee:
   print(ee)
   pass;
x = x + 1

So, it didn't worked because the variable x wasn't defined in __main__ but in in the scope of the start_counter() function and dill.load_session() restores the stuff in __main__.

And what does that __main__ stuff means?

Let's see that using a Repl:

~/$ python
Python 3.8.10 (tags/v3.8.10:3d8993a, May  3 2021, 11:48:03) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']

Perfect. We have an empty python interpreter. That dir() shows what we have in __main__.

Now we'll load some libraries, and assign a variable, and define a function just because we can:

>>> import pandas, numpy, dill, pickle, json, datetime
>>> foo = "bar"
>>> def functionWithUglyName():
...    print("yep")
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'datetime', 'dill', 'foo', 'functionWithUglyName', 'json', 'numpy', 'pandas', 'pickle']```

Well. That __main__ stuff looks more populated.

Now let's save the session and exit the Repl:

>>> dill.dump_session('session_01')
>>>  exit()

What happens when we load the session with `dill.load_session()' ?

Let's open another Repl to discover it:

>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__']

Ok. Just another empty python interpreter...

Let's load the session and see what happens:

>>> import dill
>>> dill.load_session('session_01')
>>> dir()
['__annotations__', '__builtins__', '__doc__', '__loader__', '__name__', '__package__', '__spec__', 'datetime', 'dill', 'foo', 'functionWithUglyName', 'json', 'numpy', 'pandas', 'pickle']

It loaded the contents of __main__ as expected. Wait a second. It loaded the functionWithUglyName we defined before. Is it real?

>>> functionWithUglyName()
yep

Turns out that dill is really good at serializing stuff. Most of the time you'll just need to pickle some data, but dill can do much more... and it is great for debugging and testing.

Iñigo González
  • 3,735
  • 1
  • 11
  • 27
  • the code call `load_session` on start of `count` function which should have restored `__main__` but still x is undefined why that's the case? – Chandan Nov 04 '21 at 18:14
  • @Chandan the `x` variable is not yet defined in the right-hand side of the assignment operator when you reach `x = x + 1` - the problem is the code assumes `dill.load_session()` will load and define it, but `x` is a local variable from another function: `dill.load_session()` won't define and load it (is not in `__main__`). – Iñigo González Nov 05 '21 at 08:25
  • 2
    I'm the `dill` author. Good answer. When you mention "use a database"... I'd recommend `klepto` as a potential database solution, if you go that route. It can use `dill` to store objects in several types of database, file, etc. – Mike McKerns Nov 05 '21 at 22:23