Is it possible to use a Jinja2 template to reverse engineer the parameters from a document?

Question

The general workflow of Jinja2 is params + Jinja2 template = generated document.

from jinja2 import Template
t = Template("Hello {{ something }}!")
t.render(something="World")
>>> u'Hello World!'

Is it possible to use a Jinja2 template to reverse engineer the parameters from a document? In other words I am looking for the following : Jinja2 template + generated document = params.

from jinja2 import Template
t = Template("Hello {{ something }}!")
t.reverse("Hello World!")
>>> {"something" : "World"}

The json output is not a requirement, but it would be handy.

If not what is a good approach to create logic like this?

Context: I use Jinja2 to generate Cisco Switch configuration files, and it would be a nice feature to be able to pull up documents generated in the past, and instead of showing the 1000+ line config scripts, I would like to list just the parameters. I know it can be solved by storing all params in a simple DB, but currently i don't have a DB set up, and I would avoid it, if possible.

Check out [this](https://stackoverflow.com/questions/8260490/how-to-get-list-of-all-variables-in-jinja-2-templates) and [this](https://stackoverflow.com/questions/3398850/how-to-get-a-list-of-current-variables-from-jinja-2-template) threads. Is this what you are searching for? — vrs, Jun 14 '16 at 13:37
@vrs Thanks for the quick response, I have seen those threads already. I have edited my question with some examples it might help with understanding my concrete case. — Balázs Hajdu, Jun 14 '16 at 13:40
I don't think that this is generally possible. What is your usecase? — syntonym, Jun 14 '16 at 13:42
A `diff` on the template and the rendered template would reveal the substitutions. — totoro, Jun 14 '16 at 13:46
I use Jinja2 to generate Cisco Switch configuration files, and it would be a nice feature to be able to pull up documents generated in the past, and instead of showing the 1000+ line config scripts, I would like to list just the parameters. I know it can be solved by storing all params in a simple DB, but currently i don't have a DB set up, and I would avoid it. — Balázs Hajdu, Jun 14 '16 at 13:46
Do you use any control flow like if or for loops or are you only using simple `{{ value }}` substitutions? — syntonym, Jun 14 '16 at 13:48
Generally you can't uniquely get the values back e.g. `{% if value %}True{% else %}True{% endif %}` would create the same output. Surely there are more examples that make more sense than this one. I would suggest saving the parameters as comments in the config files. — syntonym, Jun 14 '16 at 13:58

djmac · Answer 1 · 2022-07-30T12:04:50.160

In case anyone's still interested / following this question I think the library (released in 2021) basically does this job (or could be used to do the job):

parse() is the opposite of format()

https://pypi.org/project/parse/

Can define a pattern (like a Template) and feed it strings to return parameters.

Minimum working example based on the original question:

import parse
pattern = parse.compile("Hello {something}")
result = pattern.parse("Hello world")
result["something"]
>>> 'world'

score 0 · Answer 2 · edited Sep 07 '21 at 20:52

This works, but it just extracts all the parameters. It doesn't give you the order, or how they are related:

import textfsm
import tempfile
import re
import pprint

j2template = open(<template_file_path>)

find_txt = '(\{\{.+?\}\})'
all_variables = re.findall(find_txt, j2template.read())
variable_set = set()

for variable in all_variables:
    variable_set.add( variable )
    
value_block_txt = ''
for value in variable_set:
    value = value.replace('.', '')
    value_block_txt += "Value List {val_text} (\S+)\n".format(val_text = value.strip('{}\ '))

fsm_regex_block = '''
Start
'''
j2template = open(<filepath>)
for line in j2template.readlines():
    replace_list = [var for var in variable_set if(var in line)]
    if len(replace_list) > 0: 
        for string in replace_list:
                line = line.replace(string, "${"+string.replace('.', '').strip('{}. ')+"}") 
        line = "  ^"+line
        fsm_regex_block += line
        
textfsm_template = value_block_txt
textfsm_template += fsm_regex_block

f = open(<temp_file_path>, 'w+')
f.write(textfsm_template)
f.close()

fsm = textfsm.TextFSM(open(<temp_file_path>))
original_variables_list_of_lists = fsm.ParseText(jinja_device_config_output)

print(original_variables_list_of_lists)

os.unlink(f)

Outputs a list of lists like:

[
    [
        ['1', '1', '1'],
        ['12'],
        ['abcd.1234.0000'],
        ['1', '1', '12'],
        [
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V', 
            'CGNLTEST123V'
            ],
        [
            'fe81:::::/127'
        ]
    ]
]

You could then de-dupe each variable's list of input parameters to get the original values.

score 0 · Answer 3 · answered Sep 01 '21 at 09:46

this works but it doesn't give all the information

import textfsm
import tempfile
import re
import pprint

j2template = open(<template_file_path>)

find_txt = '(\{\{.+?\}\})'
all_variables = re.findall(find_txt, j2template.read())
variable_set = set()

for variable in all_variables:
    variable_set.add( variable )

value_block_txt = ''
for value in variable_set:
    value = value.replace('.', '')
    value_block_txt += "Value List {val_text} (\S+)\n".format(val_text = value.strip('{}\ '))

fsm_regex_block = '''
Start
'''
j2template = open(<filepath>)
for line in j2template.readlines():
    replace_list = [var for var in variable_set if(var in line)]
    if len(replace_list) > 0: 
        for string in replace_list:
                line = line.replace(string, "${"+string.replace('.', '').strip('{}. ')+"}") 
        line = "  ^"+line
        fsm_regex_block += line

textfsm_template = value_block_txt
textfsm_template += fsm_regex_block

f = open(<temp_file_path>, 'w+')
f.write(textfsm_template)
f.close()

fsm = textfsm.TextFSM(open(<temp_file_path>))
original_variables_list_of_lists = fsm.ParseText(jinja_device_config_output)

print(original_variables_list_of_lists)

os.unlink(f)
Outputs a list of lists like: [[['1', '1', '1'], ['12'], ['abcd.1234.0000'], ['1', '1', '12'], ['CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V', 'CGNLTEST123V'], ['fe81:::::/127']]]

then i would de-dup each variable's list of input parameters for og values.

score 0 · Answer 4 · answered Sep 10 '21 at 09:54

This open source project seems to provide the desired functionality: https://github.com/yvdlima/pytemplate-reverse

Usage example:

from template_reverse import ReverseTemplate

segments = [
    "shrek3_0_600.avi",
    "shrek3_1_560.avi",
    "shrek3_2_780.avi"
]
rt = ReverseTemplate("{video_name}_{segment_id}_{segment_duration_in_secs}.avi")

total_duration = 0

for segment in segments:
    values = rt.reverse(segment)
    print("Checking out movie", values["video_name"], "part ", values["segment_id"])
    total_duration += int(values["segment_duration_in_secs"])

print("Total video duration so far", total_duration)

Is it possible to use a Jinja2 template to reverse engineer the parameters from a document?

4 Answers4