Where to run collectstatic when deploying django app to heroku using docker?

Question

I am deploying a Django app to Heroku using Docker. When I put RUN manage.py collectstatic --noinput in the Dockerfile, it fails because there is no value set for the environment variable DJANGO_SECRET_KEY. My understanding is that this is because config vars aren't available during build time.

When I run collectstatic as a release command, it works without error, and successfully copies the static files. However, when I hit the app url, it returns a 500 error because the static files can't be found. I believe this is because the release command is run as a dyno on an ephemeral filesystem, and the copied files are therefore not found.

It seems to be a catch-22. Putting collectstatic in the Dockerfile fails because there are no config variables available, but putting it as a release command fails because only file changes from the build phase are saved?

What to do?

Here are my collectstatic settings in settings.py


MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',
    ...
]
...
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATIC_URL = '/static/'
STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'static'),
)
STATICFILES_STORAGE = 'backend.storage.WhiteNoiseStaticFilesStorage'

Dockerfile

# Pull base image
FROM python:3.7-slim

# Set environment varibles
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Set work directory
RUN mkdir /code
WORKDIR /code

# Install dependencies
RUN pip install pipenv
COPY Pipfile Pipfile.lock /code/
RUN pipenv install --system

# Copy project
COPY . /code/

## collect static files
RUN mkdir backend/staticfiles

# This fails because DJANGO_SECRET_KEY can't be empty
RUN python manage.py --noinput

heroku.yml

build:
  docker:
    web: Dockerfile
run:
  web: gunicorn backend.config.wsgi:application --bind 0.0.0.0:$PORT

They say it runs automatically during a build; https://devcenter.heroku.com/articles/django-assets#collectstatic-during-builds Also you could set the settings module in your dockerfile; `ENV DJANGO_SETTINGS_MODULE=project.settings.build` — markwalker_, Jan 13 '20 at 15:08
It doesn't automatically run when you are building a Docker container -- the link applies only if you aren't using Docker. — Ryan Knight, Jan 13 '20 at 15:12

score 35 · Accepted Answer · answered Jan 15 '20 at 19:37

35

After confirming with Heroku support, this does indeed appear to be a bit of a catch-22.

The solution was to put collectstatic in the Dockerfile so that it runs during build time and the files persist.

We got around not having a secret key config var by setting a default secret key using the get_random_secret_key function from Django.

The run phase uses the secret key from the Heroku config vars, so we aren't actually changing the secret key every time -- the default only applies to the build process. collectstatic doesn't index on the secret key, so this is fine.

In settings.py

from django.core.management.utils import get_random_secret_key
...
SECRET_KEY = os.getenv('DJANGO_SECRET_KEY', default=get_random_secret_key())

answered Jan 15 '20 at 19:37

Ryan Knight

1,288
1
11
18

6

Thank you so much! I've wasted hours trying to get static files to work on Heroku! This is a wonderfully elegant solution. – Trey Piepmeier Jun 06 '20 at 19:14
Oh man I wish I had found that post 2 days ago ! – Francois Jun 20 '21 at 13:16
Off topic: does it make sense to run `collectstatic` at all when inside a docker? – shallowThought Jun 09 '22 at 14:44
I also initially solved it this way, but my situation changed because I was trying to serve static files from S3 bucket and for that I needed additional ENV variables since the `collectstatic` needs to actually upload the static files to the bucket. Therefore the docker file gets bloated by many ARG and ENV lines and I need to do a bash for loop loading the variables from `.env` and passing them to `--build-args`. I do not like this at all. I will have to come up with smt else. Any ideas? Maybe the solution is indeed to run `collectstatic` just before the build so the bucket is up to date. – Paloha May 24 '23 at 15:50
@shallowThought Why do you think it has no sense? – David Dahan Jun 15 '23 at 00:25

markwalker_ · Answer 2 · 2020-01-13T16:59:53.863

8

I don't use heroku so can't test, but you should be able to run collect static before you run the app;

Dockerfile

# Pull base image
FROM python:3.7-slim

# Set environment varibles
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Set work directory
WORKDIR /code/

# Install dependencies
RUN pip install pipenv
COPY Pipfile Pipfile.lock .
RUN pipenv install --system

# Copy project
COPY . .

# Collect static files
RUN python manage.py collectstatic --noinput

# run gunicorn
CMD gunicorn hello_django.wsgi:application --bind 0.0.0.0:$PORT

You could also not run collectstatic in your dockerfile, or event run the application because these can be ran by heroku.yml, for example;

build:
  docker:
    web: Dockerfile
  config:
    DJANGO_SETTINGS_MODULE: project.settings
run:
  web: gunicorn backend.config.wsgi:application --bind 0.0.0.0:$PORT
release:
  image: web
  command:
    - python manage.py collectstatic --noinput

You also shouldn't need to mkdir for your working directory. Just set WORKDIR /code/ early in your dockerfile and after that things will run based on that directory.

There's a decent article on this here; https://testdriven.io/blog/deploying-django-to-heroku-with-docker/

edited Jan 13 '20 at 16:59

answered Jan 13 '20 at 15:27

markwalker_

12,078
7
62
99

3

With collectstatic in the Dockerfile the build fails because the django secret key is not available at build time for the collectstatic command. With the collectstatic as an release command it will build and release successfully but can't actually find the static files, I believe because files created during release don't persist. That article is a puzzle; I'm wondering if the author never actually tested it. – Ryan Knight Jan 13 '20 at 15:35
@RyanKnight But you'll notice I've shown the addition of the settings module as an environment variable which would allow django to load the settings & execute the command. – markwalker_ Jan 13 '20 at 15:54
2

Yup, tried that, doesn't work. The problem is that config vars aren't available at build time on Heroku: https://devcenter.heroku.com/articles/build-docker-images-heroku-yml#release-configuring-release-phase – Ryan Knight Jan 13 '20 at 16:06
@RyanKnight how about passing config variables from `heroku.yml` for the build? (I've updated my answer to include this) – markwalker_ Jan 13 '20 at 16:58
The problem when running as part of the build process isn't finding the settings file; I've got a default path set for that in `manage.py` so that's never been an issue. The problem is accessing environment variables that are set through Heroku config vars. The one config var that fails with collectstatic is the django secret key, because it returns "", but a value is needed. – Ryan Knight Jan 13 '20 at 17:08
@markwalker_ I followed the article set up and was getting the same problem. the release command `collectstatic` was not persisting the staticfiles folder of my project after release. @Ryan Knight how did later get around this? thanks. – oyeyipo Mar 28 '21 at 12:21
For those running into an issue still that is just because your project needs a SECRET_KEY set and if you are using environment variables, that is not the case with this provided snippet. You can however just use a better architecture and avoid that problem entirely though. Option 1 is to create a temporary and false key (not recommended) or you can use something like AWS SSM where it is immediately available :) – Chance Jun 21 '22 at 17:29

score 0 · Answer 3 · answered Feb 13 '23 at 13:03

0

You can prefix commands with dummy environment variables

RUN DJANGO_SECRET_KEY=secret python manage.py collectstatic --no-input

answered Feb 13 '23 at 13:03

Neil

8,925
10
44
49

Where to run collectstatic when deploying django app to heroku using docker?

3 Answers3

Linked