10
import jinja2 
from jinja2 import Template

records = [{'a':1,'b':1, 'c':1},{'a':1,'b':1, 'c':1}, {'a':2,'b':1, 'c':1}, {'a':3,'b':1, 'c':1}]    

t = jinja2.Template("""
{% set record_info = dict() %}
{% for item in records %}
{% set key =  str(item['a'])+str(item['b'])+str(item['c']) %}
{% if key in record_info %}
{% set record_info.key += 1 %}
{% else %}
{% set record_info.key = 1 %}
{% endif %}
{% endfor %}
{{record_info}}""")

This gives me:

Traceback (most recent call last):
  File "<stdin>", line 11, in <module>
  File "/Library/Python/2.7/site-packages/jinja2/environment.py", line 945, in __new__
    return env.from_string(source, template_class=cls)
  File "/Library/Python/2.7/site-packages/jinja2/environment.py", line 880, in from_string
    return cls.from_code(self, self.compile(source), globals, None)
  File "/Library/Python/2.7/site-packages/jinja2/environment.py", line 591, in compile
    self.handle_exception(exc_info, source_hint=source_hint)
  File "/Library/Python/2.7/site-packages/jinja2/environment.py", line 780, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "<unknown>", line 6, in template
jinja2.exceptions.TemplateSyntaxError: expected token 'end of statement block', got '.'

What am I doing wrong ?

Ankur Agarwal
  • 23,692
  • 41
  • 137
  • 208

3 Answers3

6

I was able to do it like this, any improvements?

records = [{'a':1,'b':1, 'c':1},{'a':1,'b':1, 'c':1}, {'a':2,'b':1, 'c':1}, {'a':3,'b':1, 'c':1}]    
t = jinja2.Template("""
{% set record_info = dict() %}
{% for item in records %}
{% set key =  item['a'] ~ ':' ~ item['b'] ~ ':' ~ item['c'] %}
{% if key in record_info %}
{% set _dummy = record_info.update( {key: record_info[key]+1 })  %}
{% else %}
{% set _dummy = record_info.update({ key:1 }) %}
{% endif %}
{% endfor %}
{{record_info}}""")    
print t.render(records=records)

{u'1:1:1': 2, u'3:1:1': 1, u'2:1:1': 1}
Ankur Agarwal
  • 23,692
  • 41
  • 137
  • 208
4

You can use "set".

{% set x = my_dict.__setitem__("key", "value") %}

And then you can use my_dict, which will have the updated value (don't mind the x, it's just there so that it won't throw an error). This is in another answer already, in here.

Yamil Abugattas
  • 408
  • 3
  • 20
  • 3
    When I try this in Jinja2 (underneath DBT) I get: `access to attribute '__setitem__' of 'dict' object is unsafe.` – dsz Oct 03 '19 at 23:34
2

Quick Solution

The existing answer works just fine, but since you asked for any improvements I will try.

from jinja2 import Template

records = [{'a':1,'b':1, 'c':1},{'a':1,'b':1, 'c':1}, {'a':2,'b':1, 'c':1}, {'a':3,'b':1, 'c':1}]

t = Template("""
{%- set record_info = dict() %}
{%- for item in records %}
  {%- set key = item['a'] ~ ':' ~ item['b'] ~ ':' ~ item['c'] %}
  {%- do record_info.update({key: 1 if not record_info[key] else record_info[key]+1}) %}
{%- endfor -%}
{{record_info}}""",
extensions=['jinja2.ext.do']
)

print(t.render(records=records))

Improvements

  • Show the imports - some SO users coming here might be brand new to Python and need full context
  • Use Python3 and the print() function (Not a criticism - I realise this is back in 2017, although from __future__ would have worked too )
  • Allow use of the do "expression statement" extension
  • Use do instead of setting the _dummy variable
  • Avoid all the extra rendered newlines/whitespace with {%- and -%} adding the - into those begin/end tags

Other changes

Whether or not these are improvements is subjective - is readability improved or hindered?

  • use inline if (ternary) to enable removal of the else clause

Notes

Using default value

This line:

{%- do record_info.update({key: 1 if not record_info[key] else record_info[key]+1}) %}

Could also be written using default:

{%- do record_info.update({key: (record_info[key] | default(0)) + 1 }) %}

Note that the check on the existing record_info[key] pipes to the default value even though it is an error when the key does not yet exist.

For example, this does not work:

{%- do record_info.update({key: (record_info[key]+1 | default(1)) }) %}

...because when it tries to evaluate record_info[key]+1 the record_info[key] error is not handled.

Environment and Loaders

The docs recommend using a Loader e.g. PackageLoader which enables template inheritance amongst other things. In your example it looks like a quick set-up so I'll ignore that requirement. Note using the Template constructor directly will create a shared environment anyway without the benefit of the Loader or inheritance.

If you use the PackageLoader you would have a package name and a folder called templates is used to load them. It's also nice to have your templates in separate files.

For a quick way to use a real loader without having those separate files, the FunctionLoader is useful. You could add more template strings to the function and return them by name, e.g.

from jinja2 import Environment, FunctionLoader

records = [{'a':1,'b':1, 'c':1},{'a':1,'b':1, 'c':1}, {'a':2,'b':1, 'c':1}, {'a':3,'b':1, 'c':1}]

def my_template(name: str) -> str:
    if name == 'first_one':
        return """
{%- set record_info = dict() %}
{%- for item in records %}
{%- set key = item['a'] ~ ':' ~ item['b'] ~ ':' ~ item['c'] %}
{%- do record_info.update({key: (record_info[key] | default(0)) + 1}) %}
{%- endfor -%}
{{record_info}}"""
    else:
        return """
"""

jinja_env = Environment(
    loader=FunctionLoader(my_template),
    extensions=['jinja2.ext.do']
)

t = jinja_env.get_template('first_one')

print(t.render(records=records))

I know it's just setting up a quick & dirty example for experimenting, but I think this is a better boilerplate starting point because it explicitly uses the jinja Environment and Loaders so can be extended more easily, and when I inevitably come back to my own answer in a few years when I forget all this I will appreciate it :D

Davos
  • 5,066
  • 42
  • 66