App engine default Django version change

Question

Since app engine 1.4.2 was released, I am getting warnings like this in my production logs:

You are using the default Django version (0.96). The default Django version will change in an App Engine release in the near future. Please call use_library() to explicitly select a Django version. For more information see http://code.google.com/appengine/docs/python/tools/libraries.html#Django

This occurs on every handler where I use a Django template - via the following:

from google.appengine.ext.webapp import template

I'd like to upgrade to 1.2, however the following links don't seem very clear on exactly how to do this (or whether it works at all):

• http://code.google.com/appengine/docs/python/tools/libraries.html#Django
• http://code.google.com/p/googleappengine/issues/detail?id=1758
• http://code.google.com/p/googleappengine/issues/detail?id=4489
• http://www.mediacrafters.org/post/django11-on-appengine

The common thread is to insert this:

from google.appengine.dist import use_library
use_library('django', '1.2')

However, in what file(s) should this be inserted:

1. Just in appengine_config.py?
2. In every .py file which does from google.appengine.ext.webapp import template?
3. In every .py file in the project?
4. In 1 and (2 or 3) above, and also add import appengine_config to those files?
5. In 3 or 4, and also add wrappers around built-in functions like appstats, remote api, datastore admin, etc?
6. Something else?

Thanks.

Answer 1

As described by Nick in the comments of systempuntoout's answer, I inserted this use_library() code from here in every handler that imports django (either directly or via google.appengine.ext.webapp.template or even just django.utils.simplejson):

from google.appengine.dist import use_library
use_library('django', '1.2')

As suggested by Nick, this was made easier by first refactoring to minimise the number of handlers referenced by app.yaml (ie, closer to scenario 1 described here).

However, I have the appstats builtin configured, and if I first went to /_ah/appstats after an upload, then I would get this error:

<'google.appengine.dist._library.UnacceptableVersionError'>: django 1.2 was requested, but 0.96.4.None is already in use

I was able to fix this by also including the use_library() code in appengine_config.py.

I noticed that by inserting a call to use_library() in appengine_config.py, then it was no longer necessary in all of my handlers. In particular the ones which import google.appengine.ext.webapp.template don't need it, because importing webapp.template loads appengine_config.py. The appstats UI imports webapp.template, which is why this fixed that problem.

However, I had some handlers (eg json services) which don't import webapp.template, but do import django.utils.simplejson. These handlers still require a direct call to use_library(). Otherwise, if those handlers are called first on a new instance, the UnacceptableVersionError occurs. Although I am using appengine_config.py to configure appstats, meaning appengine_config.py gets called to instrument all requests, it gets called too late in the page lifecycle to properly configure the correct version of Django.

This all appeared to work okay at first, but then I discovered a backwards incompatibility between the new Django 1.2 and the old Django 0.96 which I'd been using. My project structure is like this:

root
+- admin
|  +- page_admin.html
+- page_base.html

With Django 0.96, having the following in page_admin.html worked fine:

{% extends "../page_base.html" %}

With Django 1.2, I got this error:

TemplateDoesNotExist: ../page_base.html

The change in Django 1.2 seems to be that by default, Django doesn't allow loading templates which are above the original template's directory.

A workaround for this is described here, but this approach couldn't work for me, as it requires the templates to be in a templates subdirectory.

The solution to this is to set up a settings.py file, set the TEMPLATE_DIRS setting to the project root directory, and then change the extends tag to just reference "page_base.html", as described here. However, I ran into two problems trying to do this.

I was using the recommended code to render my template, ie:

template_values = { ... }
path = os.path.join(os.path.dirname(__file__), 'page_admin.html')
self.response.out.write(template.render(path, template_values))

The first problem is that template.render() overrides the TEMPLATE_DIRS setting, to set it to the directory of the template being rendered. The solution to this is the following code:

template_values = { ... }
path = os.path.join(os.path.dirname(__file__), 'page_admin.html')
template_file = open(path) 
compiled_template = template.Template(template_file.read()) 
template_file.close()  
self.response.out.write(compiled_template.render(template.Context(template_values))) 

One downside of this approach though is that template.render() caches the compiled templates, whereas this code doesn't (although that wouldn't be hard to add).

To configure the TEMPLATE_DIRS setting, I added a settings.py to my project:

PROJECT_ROOT = os.path.dirname(__file__) 
TEMPLATE_DIRS = (PROJECT_ROOT,)

And then in all of my handlers, before the use_library() code, I set the DJANGO_SETTINGS_MODULE as described here:

import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings' 

The second problem was that this didn't work - the settings file wasn't getting loaded, and so the TEMPLATE_DIRS was empty.

Django settings are loaded from the specified settings.py lazily, the first time they are accessed. The problem is that importing webapp.template calls django.conf.settings.configure() to attempt to set up some settings. Therefore if webapp.template is imported before any settings are accessed, then settings.py is never loaded (as the settings accessor finds that settings already exist, and doesn't attempt to load any more).

The solution to this is to force an access to the settings, to load the settings.py, before webapp.template is imported. Then when webapp.template is later imported, its call to django.conf.settings.configure() is ignored. I therefore changed the Django version setup code in all of my handlers (and appengine_config.py) to the following:

import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings' 

from google.appengine.dist import use_library
use_library('django', '1.2')

from django.conf import settings
_ = settings.TEMPLATE_DIRS

In practise, I actually put all of the above code in a file called setup_django_version.py, and then import that from all of my handlers, rather than duplicating these 6 lines of code everywhere.

I then updated my page_admin.html template to include this (ie specify page_base.html relative to the TEMPLATE_DIRS setting):

{% extends "page_base.html" %}

And that fixed the problem with rendering the admin page.

Answer 2

As of GAE 1.5.0, there's much simpler, though momentarily under-documented, way of specifying which version of Django templates you want to use.

In appengine_config.py, include the line

webapp_django_version = '1.2'

That's it.

No more need for use_library().

Answer 3

According to the documentation you are properly linking, you should just add this function at the beginning of your main.py script handler .

Answer 4

One thing I'd like to mention that the documentation doesn't make clear: if you use google.appengine.ext.deferred and have use_library in your main.py, then when the deferred task is executed it will NOT load main.py and if you are unlucky enough to have a deferred task as your first request to an instance, it will bork the instance (causing it to throw UnacceptableVersionError when your main.py attempts to call use_library on a later request). I think if you add use_libary to appengine_config.py it will work with deferred as well, but we ended up switching to regular task queues (which handlers ARE routed through main.py) to avoid this problem.