10

I am trying django-pipeline in order to minify static resources, use cache for them and make my templates simpler. My CSS and JS files are found and loaded by my browser but it takes about 10 seconds for my (very simple) home page to load.

enter image description here

I am using Python 2.7.6, Django 1.7.3 and django-pipeline 1.4.3. PyCharm runs the development server with a local virtualenv.

My settings.py contains the following:

DEBUG = True
TEMPLATE_DEBUG = DEBUG

INSTALLED_APPS = (
    'django_admin_bootstrapped', # custom admin
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    # pip installed apps
    'pipeline',
    # project apps
    'myapp',
)

MIDDLEWARE_CLASSES = (
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.auth.middleware.SessionAuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'pipeline.middleware.MinifyHTMLMiddleware',
)

STATICFILES_FINDERS = (
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
    'pipeline.finders.FileSystemFinder',
    'pipeline.finders.CachedFileFinder',
    'pipeline.finders.PipelineFinder',
)

STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'

STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, 'static')

STATICFILES_DIRS = (
    os.path.join(BASE_DIR, 'myapp/static'),
)

STATICFILES_STORAGE = 'pipeline.storage.PipelineCachedStorage'
PIPELINE_CSS_COMPRESSOR = 'pipeline.compressors.yuglify.YuglifyCompressor'
PIPELINE_JS_COMPRESSOR = 'pipeline.compressors.yuglify.YuglifyCompressor'

PIPELINE_CSS = {
    'base': {
        'source_filenames': (
            'myapp/css/base.css',
            'myapp/bower_components/bootstrap/dist/css/bootstrap.css',
            'myapp/bower_components/Hover/css/hover.css',
            'myapp/bower_components/font-awesome/css/font-awesome.css',
        ),
        'output_filename': 'css/myapp.css',
    },
}

PIPELINE_JS = {
    'base': {
        'source_filenames': (
            'myapp/bower_components/jquery/dist/jquery.min.js',
            'myapp/bower_components/bootstrap/dist/js/bootstrap.min.js',
        ),
        'output_filename': 'js/myapp.js',
    },
}

My base HTML template contains the following:

{% load staticfiles %}
{% load pipeline %}

<!DOCTYPE html>
<html>
    <head>
        [...]
        {% block css %}
            {% stylesheet 'base' %}
        {% endblock css %}

        {% block javascript %}
            {% javascript 'base' %}
        {% endblock javascript %}

    </head>
    <body> [...] </body>
</html>

My home.html extends base.html but does not use the css nor javascript pipeline's template tags.

Just to make sure yuglify is available:

$ yuglify --version
0.1.4

What am I doing wrong here?

Note: browser does not find static assets (myapp.css and myapp.js) if PIPELINE_ENABLED = True.

M.javid
  • 6,387
  • 3
  • 41
  • 56
Q Caron
  • 952
  • 13
  • 26
  • Any luck figuring this out? I am having the same problem when using pipeline locally with S3. ie, It happens when I am using DEBUG=True (and PIPELINE_ENABLED=False, which is default). With pipeline enabled, it works as expected. – John Lehmann Jul 28 '15 at 12:35
  • 2
    From what I remember, the page was loading really slowly because the file finders had a huge path tree to go through. The problem was that with debug=False in development, you have to use specific finders : https://github.com/cyberdelia/django-pipeline/issues/418. I think I ended up at this point when point when I created this topic. Because I manage front-end packages with bower, all directories (src, dist etc.) and files (.json or other useless files) get included in the static files! – Q Caron Jul 28 '15 at 15:07
  • 1
    Please see https://github.com/cyberdelia/django-pipeline/issues/482 for more information. – Q Caron Jul 29 '15 at 07:25
  • What domain do you happen to be running on? I oddly had something like that when I was using `dockerhost:8000`, and it mysteriously went away when I changed back to `localhost:8000`. Sometimes I wonder if `localhost` doesn't have even higher elevated privileges than just being a default entry in hosts files. – Craig Labenz Sep 06 '15 at 17:50

1 Answers1

2

The problem is the template tag code is doing a bunch of stuff including running collectstatic for every request when debug is True which makes dev painfully slow. Even if debug is False the templatetag will still connect to and query S3 for several things. When the files are local this is not a (big) problem but when using S3 it is. The only solution I could come up with is writing my own simplified templatetag called pipelines.py.

Before you get into it there is two important things you need to know, first to get pipeline to work I have an empty shell S3PipelineStorage which combines pipeline and boto, you probably already have this if you have s3 + pipeline working but it is important:

from pipeline.storage import PipelineMixin
from storages.backends.s3boto import S3BotoStorage

class S3PipelineStorage(PipelineMixin, S3BotoStorage):
    pass

Then in settings:

STATICFILES_STORAGE = 'path.to.your.file.S3PipelineStorage'

Now, if you look down at the templatetag you will see I use staticfiles_storage.url similar to the original templatetag. This adds the s3 path to the relative path but if you don't add this setting you will query S3 every time to generate the URL. You can add the setting or just hard code your s3 path instead of staticfiles_storage.url however I suggest you add the setting because it will improve performance anywhere a URL is generated for an s3 resource.

AWS_S3_CUSTOM_DOMAIN = 'your_bucket-%s.s3.amazonaws.com' % ENVIRONMENT.lower()

Now you are ready for the templatetag. To use it simply {% load pipelines %} instead of {% load pipeline %}.

from django.contrib.staticfiles.storage import staticfiles_storage
from django import template
from django.template.loader import render_to_string
from django.utils.safestring import mark_safe
from pipeline.conf import settings

register = template.Library()

@register.simple_tag
def stylesheet(group):

    if group not in settings.PIPELINE_CSS:
        return ''

    if settings.DEBUG is False or settings.PIPELINE_ENABLED is True:
        context = {
            'type': 'text/css',
            'url': mark_safe(staticfiles_storage.url(settings.PIPELINE_CSS[group]['output_filename']))
        }
        html = render_to_string("pipeline/css.html", context)
    else:
        html = ''
        for path in settings.PIPELINE_CSS[group]['source_filenames']:
            context = {
                'type': 'text/css',
                'url': mark_safe(staticfiles_storage.url(path))
            }
            html = "%s\n        %s" % (html, render_to_string("pipeline/css.html", context))

    return html

@register.simple_tag
def javascript(group):

    if group not in settings.PIPELINE_JS:
        return ''

    if settings.DEBUG is False or settings.PIPELINE_ENABLED is True:
        context = {
            'type': 'text/javascript',
            'url': mark_safe(staticfiles_storage.url(settings.PIPELINE_JS[group]['output_filename']))
        }
        html = render_to_string("pipeline/js.html", context)
    else:
        html = ''
        for path in settings.PIPELINE_JS[group]['source_filenames']:
            context = {
                'type': 'text/javascript',
                'url': mark_safe(staticfiles_storage.url(path))
            }
            html = "%s\n        %s" % (html, render_to_string("pipeline/js.html", context))

    return html
dotcomly
  • 2,154
  • 23
  • 29