Unable to Include Jinja2 Template to Pyinstaller Distribution

Question

I have a python script that uses Jinja2 template, and I'm trying to create a one-folder distribution using Pyinstaller.

In Jinja, I'm letting the program understand the location of the templates by using a PackageLoader class. The code below shows that it's pointing to my templates folder under pycorr Python package.

env = Environment(loader=PackageLoader('pycorr', 'templates'))
template = env.get_template('child_template.html')

And here's what my folder structure looks like:

pycorr
| |
| + templates
|    |
|    + base.html
|    + child.html

When I compile the package into a single folder using Pyinstaller, I don't see any warning/error related to Jinja2, and I'm able to start the .exe file. However when the program start to look for Jinja2 template, it fails with this error message displayed on the console window:

Traceback (most recent call last):
  ...
  File "C:\Users\ ... \out00-PYZ.pyz\pycorr.WriterToHTML", line 96, in htmlout_table
  File "C:\Users\ ... \out00-PYZ.pyz\pycorr.WriterToHTML", line 13, in __init__
  File "C:\Users\ ... \out00-PYZ.pyz\pycorr.WriterToHTML", line 48, in __set_template
  File "C:\Users\ ... \out00-PYZ.pyz\jinja2.environment", line 791, in get_template
  File "C:\Users\ ... \out00-PYZ.pyz\jinja2.environment", line 765, in _load_template
  File "C:\Users\ ... \out00-PYZ.pyz\jinja2.loaders", line 113, in load
  File "C:\Users\ ... \out00-PYZ.pyz\jinja2.loaders", line 224, in get_source
  File "C:\Users\ ... \dist\OCA_CO~1\eggs\setuptools-14.3-py2.7.egg\pkg_resources\__init__.py", line 1572, in has_resource 
    return self._has(self._fn(self.module_path, resource_name))
  File "C:\Users\ ... \dist\OCA_CO~1\eggs\setuptools-14.3-py2.7.egg\pkg_resources\__init__.py", line 1627, in _has
    "Can't perform this operation for unregistered loader type"
  NotImplementedError: Can't perform this operation for unregistered loader type

I don't really understand the error message, but my guess is that Pyinstaller need to find the templates folder. So I added these lines in the Pyinstaller .spec file:

a.datas += [('BASE', './pycorr/templates/base.html', 'DATA')]
a.datas += [('TABLE', './pycorr/templates/table_child.html', 'DATA')]
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=None,
               upx=False,
               name='OCA_correlation')

But it doesn't seems to solve the issue.

Can anyone help? I read up the Pyinstaller manual several times but I just can't figure it out.

Answer 1

I encountered a similar Jinja2 error when trying to render a Pandas DataFrame to html from within a PyInstaller distribution using the code,

html = df.style.render()

I resolved the issue by modifying the package loader instruction.

In the Pandas style file: site-packages\pandas\io\formats\style.py

I replaced,

loader = PackageLoader("pandas", "io/formats/templates")   

With,

if getattr(sys, 'frozen', False):
    # we are running in a bundle
    bundle_dir = sys._MEIPASS
    loader = FileSystemLoader(bundle_dir)
else:
    loader = PackageLoader("pandas", "io/formats/templates")

And a corresponding import at the top of the file,

import sys

Now, if the program is 'frozen' then the loader will look for the template in the bundle directory. In which case, the final step was to add the template to the bundle. To do this I ran PyInstaller from the command line with the --add-data command. For example, a command similar to that below adds the default template html.tpl,

pyinstaller --add-data PATH1\site-packages\pandas\io\formats\templates\html.tpl;.  PATH2\Main.py

Answer 2

I ran into this problem when building a GUI use pyinstaller. I used Jinja2 to render a report and the templates did not load, instead I received "unregistered loader type" error as well. Reading and testing many solutions online I finally had a fix: FileSystemLoader has to be used instead of PackageLoader. Also a file path needs to be provided for the FileSystemLoader. My final solution is a combination of information from here and here.

The following provide a complete example for this solution. My code is under testjinjia2 with templates in sub-directory templates:

testjinja2
| |
| + templates
|    |
|    + base.html
|    + report.html
testreport.py
testreport.spec

In testreport.spec:

# -*- mode: python -*-

block_cipher = None


a = Analysis(['E:\\testjinja2\\testreport.py'],
             pathex=['E:\\testjinja2'],
             binaries=[],
             datas=[('E:\\testjinja2\\templates\\base.html', '.'),
                    ('E:\\testjinja2\\templates\\report.css', '.'),
                    ('E:\\testjinja2\\templates\\report.html', '.')],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          a.binaries,
          a.zipfiles,
          a.datas,
          name='testreport',
          debug=False,
          strip=False,
          upx=True,
      console=True )

In testreport.py,

import os
import sys
from jinja2 import Environment, PackageLoader, FileSystemLoader


def resource_path(relative_path, file_name):
    """ Get absolute path to resource, works for both in IDE and for PyInstaller """
    # PyInstaller creates a temp folder and stores path in sys._MEIPASS
    # In IDE, the path is os.path.join(base_path, relative_path, file_name)
    # Search in Dev path first, then MEIPASS
    base_path = os.path.abspath(".")
    dev_file_path = os.path.join(base_path, relative_path, file_name)
    if os.path.exists(dev_file_path):
        return dev_file_path
    else:
        base_path = sys._MEIPASS
        file_path = os.path.join(base_path, file_name)
        if not os.path.exists(file_path):
            msg = "\nError finding resource in either {} or {}".format(dev_file_path, file_path)
            print(msg)
            return None
        return file_path

class Report:

    def main(self, output_html_file):
        # template_loader = PackageLoader("report", "templates")
        # --- PackageLoader returns unregistered loader problem,  use FileSystemLoader instead
        template_file_name = 'report.html'
        template_file_path = resource_path('templates', template_file_name)
        template_file_directory = os.path.dirname(template_file_path)
        template_loader = FileSystemLoader(searchpath=template_file_directory)
        env = Environment(loader=template_loader)  # Jinja2 template environment
        template = env.get_template(template_file_name)
        report_content_placeholder = "This is my report content placeholder"
        html = template.render(report_content= report_content_placeholder)
        with open(output_html_file, 'w') as f:
            f.write(html)

if __name__ == "__main__":
    my_report = Report()
    my_report.main("output.html")

A method resource_path is needed because the file path to jinja templates files are different in my IDE and the extracted files from the exe file.

Also just some simple template files for you to try on.
base.html

<html>
<head lang="en">
    <meta charset="UTF-8">
    <title></title>

    <style>
        .centered {
            text-align: center;
        }
        .centeredbr {
            text-align: center;
            page-break-before:always;
        }

        .underlined {
            text-decoration: underline;
        }

    </style>
</head>
<body>
{% block body %}{% endblock %}
</body>
</html>

Report html

<!DOCTYPE html>
{% extends "base.html" %}

{% block body %}
<h1 class="centered underlined">Report Title</h1>

<h2 class="centeredbr">Chaper I</h2>

<p>{{ report_content }}</p>

{% endblock %}

I am using pyinstaller 3.2.1 and Python 3.5.1 Anaconda Custom (64-bit)

Answer 3

Going from @Uynix, I found that I had to do a few more steps to implement the solution for my version of the problem using cx_freeze. My first solution post so let me know if more details are needed.

In summary, I had to modify the C:\ProgramData\Anaconda3\pkgs\bokeh-0.12.9-py36_0\Lib\site-packages\bokeh\core\templates.py

The Original file (bokeh 0.12.9):

''' Provide Jinja2 templates used by Bokeh to embed Bokeh models
(e.g. plots, widgets, layouts) in various ways.

.. bokeh-jinja:: bokeh.core.templates.AUTOLOAD_JS
.. bokeh-jinja:: bokeh.core.templates.AUTOLOAD_NB_JS
.. bokeh-jinja:: bokeh.core.templates.AUTOLOAD_TAG
.. bokeh-jinja:: bokeh.core.templates.CSS_RESOURCES
.. bokeh-jinja:: bokeh.core.templates.DOC_JS
.. bokeh-jinja:: bokeh.core.templates.FILE
.. bokeh-jinja:: bokeh.core.templates.JS_RESOURCES
.. bokeh-jinja:: bokeh.core.templates.NOTEBOOK_LOAD
.. bokeh-jinja:: bokeh.core.templates.NOTEBOOK_DIV
.. bokeh-jinja:: bokeh.core.templates.PLOT_DIV
.. bokeh-jinja:: bokeh.core.templates.SCRIPT_TAG

'''
from __future__ import absolute_import

import json

from jinja2 import Environment, PackageLoader, Markup

_env = Environment(loader=PackageLoader('bokeh.core', '_templates'))
_env.filters['json'] = lambda obj: Markup(json.dumps(obj))


JS_RESOURCES = _env.get_template("js_resources.html")

CSS_RESOURCES = _env.get_template("css_resources.html")

SCRIPT_TAG = _env.get_template("script_tag.html")

PLOT_DIV = _env.get_template("plot_div.html")

DOC_JS = _env.get_template("doc_js.js")

FILE = _env.get_template("file.html")

NOTEBOOK_LOAD = _env.get_template("notebook_load.html")

NOTEBOOK_DIV = _env.get_template("notebook_div.html")

AUTOLOAD_JS = _env.get_template("autoload_js.js")

AUTOLOAD_NB_JS = _env.get_template("autoload_nb_js.js")

AUTOLOAD_TAG = _env.get_template("autoload_tag.html")

I traced the problem to the line:

JS_RESOURCES = _env.get_template("js_resources.html")

Which I discovered, somehow was not being compiled correctly by cx_freeze, throwing that same error:

  File "C:\ProgramData\Anaconda3\lib\site-packages\bokeh\core\templates.py", line 27, in <module>
    JS_RESOURCES = _env.get_template("js_resources.html")
  File "C:\ProgramData\Anaconda3\lib\site-packages\jinja2\environment.py", line 830, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "C:\ProgramData\Anaconda3\lib\site-packages\jinja2\environment.py", line 804, in _load_template
    template = self.loader.load(self, name, globals)
  File "C:\ProgramData\Anaconda3\lib\site-packages\jinja2\loaders.py", line 113, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "C:\ProgramData\Anaconda3\lib\site-packages\jinja2\loaders.py", line 234, in get_source
    if not self.provider.has_resource(p):
  File "C:\ProgramData\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 1464, in has_resource
    return self._has(self._fn(self.module_path, resource_name))
  File "C:\ProgramData\Anaconda3\lib\site-packages\pkg_resources\__init__.py", line 1514, in _has
    "Can't perform this operation for unregistered loader type"
NotImplementedError: Can't perform this operation for unregistered loader type

New templates.py file:

''' Provide Jinja2 templates used by Bokeh to embed Bokeh models
(e.g. plots, widgets, layouts) in various ways.

.. bokeh-jinja:: bokeh.core.templates.AUTOLOAD_JS
.. bokeh-jinja:: bokeh.core.templates.AUTOLOAD_NB_JS
.. bokeh-jinja:: bokeh.core.templates.AUTOLOAD_TAG
.. bokeh-jinja:: bokeh.core.templates.CSS_RESOURCES
.. bokeh-jinja:: bokeh.core.templates.DOC_JS
.. bokeh-jinja:: bokeh.core.templates.FILE
.. bokeh-jinja:: bokeh.core.templates.JS_RESOURCES
.. bokeh-jinja:: bokeh.core.templates.NOTEBOOK_LOAD
.. bokeh-jinja:: bokeh.core.templates.NOTEBOOK_DIV
.. bokeh-jinja:: bokeh.core.templates.PLOT_DIV
.. bokeh-jinja:: bokeh.core.templates.SCRIPT_TAG

'''
from __future__ import absolute_import

import json
import sys, os
import bokeh.core

# from jinja2 import Environment, PackageLoader, Markup
from jinja2 import Environment, Markup, FileSystemLoader

# add in from Uynix
def resource_path(relative_path, file_name):
    """ Get absolute path to resource, works for both in IDE and for PyInstaller """
    # PyInstaller creates a temp folder and stores path in sys._MEIPASS
    # In IDE, the path is os.path.join(base_path, relative_path, file_name)
    # Search in Dev path first, then MEIPASS
    base_path = os.path.abspath(".")
    dev_file_path = os.path.join(base_path, relative_path, file_name)
    if os.path.exists(dev_file_path):
        return dev_file_path
    else:
        base_path = sys._MEIPASS
        file_path = os.path.join(base_path, file_name)
        if not os.path.exists(file_path):
            msg = "\nError finding resource in either {} or {}".format(dev_file_path, file_path)
            print(msg)
            return None
        return file_path



""" my new code here
"""
_env = Environment(loader=FileSystemLoader(os.path.dirname(bokeh.core.__file__) +'\\_templates'))
""" end of my new code
"""


_env.filters['json'] = lambda obj: Markup(json.dumps(obj))

# this is where the errors start to happen! need to replace get_template!
JS_RESOURCES = _env.get_template("js_resources.html")

CSS_RESOURCES = _env.get_template("css_resources.html")

SCRIPT_TAG = _env.get_template("script_tag.html")

PLOT_DIV = _env.get_template("plot_div.html")

DOC_JS = _env.get_template("doc_js.js")

FILE = _env.get_template("file.html")

NOTEBOOK_LOAD = _env.get_template("notebook_load.html")

NOTEBOOK_DIV = _env.get_template("notebook_div.html")

AUTOLOAD_JS = _env.get_template("autoload_js.js")

AUTOLOAD_NB_JS = _env.get_template("autoload_nb_js.js")

AUTOLOAD_TAG = _env.get_template("autoload_tag.html")

Then ran cx_freeze etc again, and this time bokeh now works!