webapp2: automatically reload code changes

Question

I'm using webapp2 for web development. In my dev environment, how can I have it reload code changes automatically? I'm using httpserver.serve(app, host='127.0.0.1', port='8008') but every time I change my code I need to stop the server and start it again.

I've used webapp2 with the Google App Engine launcher provided by Google and I don't need to restart it every time I make a change. How do they do it? Do they monitor file system changes and call reload on modules when there is a change?

This isn't really a GAE question. The fact that you're using webapp2 is almost certainly not related to your problem. — allyourcode, Apr 13 '13 at 11:02
After my research I believe this is relevant for GAE users so I will add the tag along with the Django tag. — Josep Valls, May 23 '16 at 22:09

score 1 · Accepted Answer · edited May 23 '17 at 12:07

Turns out that the Google App Engine launcher provided by Google is borrowing the autoreload code from the Django framework, specifically this: https://github.com/django/django/blob/master/django/utils/autoreload.py

Searched Stackoverflow and there were some related questions that provide answers: How to automatically reload Django when files change? which in turn are linked to this module which is able to detect and reload changed code: https://github.com/tjwalch/django-livereload-server

If someone wants more details on how I did it.

In my launcher:

import autoreloader,thread
thread.start_new_thread(autoreloader.reloader_thread, ())
...
app = webapp2.WSGIApplication(...)
def main():
    from paste import httpserver
    httpserver.serve(app, ...)

autoreloader.py:

import sys,os,time,logging

RUN_RELOADER = True

logger = logging.getLogger(__name__)

whitelist = ['webapp2', 'paste', 'logging']

# this code is from autoreloader
_mtimes = {}
_win = (sys.platform == "win32")
_error_files = []
_cached_modules = set()
_cached_filenames = []


def gen_filenames(only_new=False):
    global _cached_modules, _cached_filenames
    module_values = set(sys.modules.values())
    _cached_filenames = clean_files(_cached_filenames)
    if _cached_modules == module_values:
        # No changes in module list, short-circuit the function
        if only_new:
            return []
        else:
            return _cached_filenames + clean_files(_error_files)

    new_modules = module_values - _cached_modules
    new_filenames = clean_files(
        [filename.__file__ for filename in new_modules
         if hasattr(filename, '__file__')])

    _cached_modules = _cached_modules.union(new_modules)
    _cached_filenames += new_filenames
    if only_new:
        return new_filenames + clean_files(_error_files)
    else:
        return _cached_filenames + clean_files(_error_files)

def clean_files(filelist):
    filenames = []
    for filename in filelist:
        if not filename:
            continue
        if filename.endswith(".pyc") or filename.endswith(".pyo"):
            filename = filename[:-1]
        if filename.endswith("$py.class"):
            filename = filename[:-9] + ".py"
        if os.path.exists(filename):
            filenames.append(filename)
    return filenames

# this code is modified from autoreloader
def check_code_changed():
    global _mtimes, _win
    for filename in gen_filenames():
        stat = os.stat(filename)
        mtime = stat.st_mtime
        if _win:
            mtime -= stat.st_ctime
        if filename not in _mtimes:
            _mtimes[filename] = mtime
            continue
        if mtime != _mtimes[filename]:
            _mtimes = {}
            try:
                del _error_files[_error_files.index(filename)]
            except ValueError:
                pass
            mname = filename.split('/')[-1].split('.')[0]
            logger.info('CHANGED %s, RELOADING %s' % (filename,mname))
            try:
                reload(sys.modules[mname])
            except:
                pass

    return False

def reloader_thread():
    while RUN_RELOADER:
        check_code_changed()
        time.sleep(1)

Comments and suggestions on my autoreloader changes welcome! — Josep Valls, May 23 '16 at 23:17
Thanks for the great example. I'm not sure if this works better on other operating systems, or it is a python version issue. I'm seeing a key error trying to use this on windows with PY3. I've updated to use threading and importlib. The sys.modules dictionary seems to require more than just the module name, but it's packages as well = e.g. "include.lib.utils" rather than "utils". — Mark, Sep 10 '19 at 23:10
This may not work on older versions of python, but worked on python3 for me. Instead of accessing the module by key in sys.modules, I've filtered the modules by file to match the changed file: `module = next(x for x in sys.modules.values() if hasattr(x,'__file__') and x.__file__ == filename) reload(module)` — Mark, Sep 10 '19 at 23:21

score 0 · Answer 2 · answered Jun 25 '20 at 12:05

Paste has its own auto-reload funcionallity (maybe it wasn't there when the original answer was provided)..

app = webapp2.WSGIApplication(...)
def main():
    from paste import httpserver, reload
    reload.install()
    httpserver.serve(app, ...)

Then just start your server using the shell script provided by Paste:

err=3
while test "$err" -eq 3 ; do
    python server.py
    err="$?"
done

Now every time you edit a file in your project the server gets auto reloaded.

Reference Docs:

webapp2: automatically reload code changes

2 Answers2