How to make Flask/Jinja2 load bundled templates in an executable zip archive?

Question

I have packaged my Flask web application into an executable Python zipped archive (zipapp). I am having problems with the loading of templates. Flask/Jinja2 is unable to find the templates.

To load templates, I used jinja2.FunctionLoader with a loading function that should have been able to read bundled files (in this case, Jinja templates) from inside the executable zip archive (reference: python: can executable zip files include data files?). However, the loading function is unable to find the template (see: (1) in the code), even though the template is readable from outside the loading function (see: (2) in the code).

Here's the directory structure:

└── src
    ├── __main__.py
    └── templates
        ├── index.html
        └── __init__.py  # Empty file.

src/__main__.py:

import pkgutil
import jinja2
from flask import Flask, render_template


def load_template(name):
    # (1) ATTENTION: this produces an error. Why?
    # Error message:
    #   FileNotFoundError: [Errno 2] No such file or directory: 'myapp'
    data = pkgutil.get_data('templates', name)
    return data

# (2) ATTENTION: Unlike (1), this successfully found and read the template file. Why?
data = pkgutil.get_data('templates', 'index.html')
print(data)
# This also works:
data = load_template('index.html')
print(data)
# Why?

app = Flask(__name__)
app.config['SECRET_KEY'] = 'my-secret-key'
app.jinja_loader = jinja2.FunctionLoader(load_template)  # <-


@app.route('/')
def index():
    return render_template('index.html')


app.run(host='127.0.0.1', port=3000)

To produce the executable archive, I installed all dependencies into src/ (using pip3 install wheel flask --target src/), then I ran python3 -m zipapp src/ -o myapp to produce the executable archive itself. I then ran the executable archive using python3 myapp. Unfortunately, trying to access the the index page through a web browser results in an error:

# ...
  File "myapp/__main__.py", line 10, in load_template
  File "/usr/lib/python3.6/pkgutil.py", line 634, in get_data
    return loader.get_data(resource_name)
FileNotFoundError: [Errno 2] No such file or directory: 'myapp'

The error is caused by (1) in the code. As part of my debugging effort, I added (2) to check whether or not the template could be found in the global scope of the file. Surprisingly, it successfully finds and reads the template file.

What accounts for the difference in behavior between (1) and (2)? More importantly, how can I make Flask find Jinja templates that are bundled together with a Flask app inside an executable Python zip archive?

(Python version: 3.6.8 on linux; Flask version: 1.1.1)

Answer 1

jinja2.FunctionLoader(load_template) is looking for a function to return the full index.html template as a unicode string. Per the jinja2 docs:

A loader that is passed a function which does the loading. The function receives the name of the template and has to return either an unicode string with the template source, a tuple in the form (source, filename, uptodatefunc) or None if the template does not exist.

pkgutil.get_data('templates', name) doesn't return a unicode string, instead it returns a bytes object. To fix this, you should use pkgutil.get_data('templates', name).decode('utf-8')

def load_template(name):
    """
    Loads file from the templates folder and returns file contents as a string.
    See jinja2.FunctionLoader docs.
    """
    return pkgutil.get_data('templates', name).decode('utf-8') 

This means that part (2) will work fine since the code is printing index.html as a bytes object. Print can handle a bytes object and it will look almost the same as a string on the console. However, the code in part (1) will fail since it is fed to jinja2.FunctionLoader which expects a string. Part (1) failed with a ValueError for me.

I suspect that since your error message is a FileNotFoundError and calls out myapp as the file, that part of your post does not exactly match your application. I replicated the instructions exactly on both Windows 10 and Ubuntu Server 18.04 as well as with Python 3.6 and 3.7 and had no issues aside from needed to use decode. I did occasionally run into PermissionErrors on Ubuntu which required me to run sudo python3 myapp.

Answer 2

Once you update your code ,you should renew you myapp. And then you will find that pkgutil.get_data('templates', name) always return the context data.

While I try your original code , it was failed because of ValueError: too many values to unpack (expected 3), which was generated by app.jinja_loader = jinja2.FunctionLoader(load_template) .

In my opinion, the best way is custom your own render_template like this:

import pkgutil
import jinja2
from flask import Flask

app = Flask(__name__)
app.config['SECRET_KEY'] = 'my-secret-key'
template = jinja2.Environment()


def load_template(name):
    data = pkgutil.get_data('templates', name)
    return data


def render_template(name, **kwargs):
    data = load_template(name).decode()
    tpl = template.from_string(data)
    return tpl.render(**kwargs)


@app.route('/')
def index():
    return render_template('index.html')


app.run(host='127.0.0.1', port=3000)

---

Besides, here are the suggestions:

1. make a new work direction, and copy my demo code as __main__.py and redo the commands to create new zipapp.

2. Don't try to print(sys.modules["templates"].__file__) before pkgutil.get_data('templates', name), the module is not loaded util pkgutil.get_data by importlib.util.find_spec

3. After pkgutil.get_data('templates', name), you maybe debug and get sys.modules["templates"].__file__ == "myapp/templates/__init__.py"

4. In the zipapp , myapp/templates/ isn't a directory, so it won't work if you try to set app.template_folder = os.path.dirname(sys.modules["templates"].__file__)