How are django templates parsed?

Question

I have been trying to learn more about Django's templating engine as it's always seemed like a bit of a black box to me. The documentation gives a good outline of the general steps involved and indicates that the template is loaded and parsed, creating a tree of nodes that are rendered (in a cascade?) with a context and appended together to give the result.

What I don't understand is the approach to parsing, and under what criteria the nodes are created? What constitutes a particular node after parsing, and how does this effect the creation of custom template tags (i.e. is there a better and more efficient way to write template tags that would lead to less nodes?).

I started reading `django.template.base` but got out as soon as I sensed powerful, time bending magic (like goto 5am). I'll upvote and check in the morning instead :) — Yuji 'Tomita' Tomita, Nov 22 '11 at 07:04

Ski · Answer 1 · 2014-10-09T12:23:57.613

A node is created out of every tag. You can get an idea how it works by reading through how to write custom tags. Anything which is inside the tag will be its children. Here is an example of a comment tag from django docs:

def do_comment(parser, token):
    nodelist = parser.parse(('endcomment',))
    parser.delete_first_token()
    return CommentNode()

as you see comment tag will parse everything until the "endcomment" and will throw it away. Other tags would pass nodelist to SometagNode() and will use it for rendering.

Rendering is done recursively. When a render() is called on node it runs render on its children and so on.

Parsing is done recursively as well this is why you can get nested tags and parser.parse() will manage to find the proper matching closing tag, because when it does parsing and stumbles on a tag it calls do_tag() thing, which in turn will call the parser.parse() again to find the nearest closing tag and will wrap everything into a node, return a node, the higher parser.parse() will put it on a node list and will continue to search for the closing tag.

Context object in nodes is a kind of list of dicts structure. Extra context is pushed on top of existing context and passed to children nodes and popped out after node is rendered so that it wouldn't effecting upper scope.

For tags which don't have children the parser.parse() is not used, and so node instance is returned without any children.

score 3 · Answer 2 · edited May 23 '17 at 12:34

3

One way of understanding more about the process is to run django with the werkzeug debugger, and trigger an exception in a template. That way, you will be able to view (and interact) with the whole stack up to that point.

edited May 23 '17 at 12:34

Community

1
1

answered Jan 21 '12 at 11:15

Marcin

48,559
18
128
201

sepulchered · Answer 3 · 2011-11-28T15:33:29.327

2

I think that first thing you should look at is code.djangoproject.com with django/template/base.py - the first (as Yuji Tomita stated before). Or download sources and look through it with your favorite editor or IDE.

edited Nov 28 '11 at 15:33

answered Nov 28 '11 at 12:08

sepulchered

814
7
18

score 0 · Answer 4 · answered Jan 28 '12 at 10:12

I guess they use tokenizing and parsing

a simple way to discribe is:

Tokenizing: break appart the code to types like:

integer foo = "bar" + 15;

this consists of

T_VARIABLETYPE  + T_VARIABLENAME  + T_EQUALS + T_STRING + T_PLUS + T_DIGIT + T_SEMI

after this you can parse by trying to find patterns with a parser

parsing:

find the pattern:

T_VARIABLETYPE  + T_VARIABLENAME  + T_EQUALS + {A recursive thing} + T_SEMI

This way you can execute a command

If you like to experiment with this i could recomend to use "ANTLR" http://www.antlr.org/ Its available in lot of different languages like Java or C# and even PHP and JS

How are django templates parsed?

4 Answers4