Serving gzipped content from django

Question

I'm trying to serve a gzipped version of a text/html page in Django, but Firefox is telling me there's a content encoding error.

NOTES:

I realize this is not a best practice and I'm most likely going to use mod_gzip. This is just a learning exercise to understand what's going on.
I know about the Django gzip middleware-- it has problems with binary files.

Here's my code:

rendered_page =  zlib.compress(template.render(context).encode('utf-8'))

response = HttpResponse(rendered_page)
response['Content-Encoding'] = 'gzip'
response['Content-Length'] = len(rendered_page)
return response

Am I missing something here? Is it possible that the content length is wrong? Are there additional headers I'm missing?

Thanks.

BTW... if you're using Apache 2.x, have a peek at mod_deflate... it's a bit more advanced than mod_zip. — Jarret Hardie, Jun 05 '09 at 20:05

score 97 · Answer 1 · edited Nov 11 '22 at 22:11

97

You could also simply use Django's GZip Middleware:

Either by enabling the middleware in settings.py by adding:

MIDDLEWARE_CLASSES = (
    "django.middleware.gzip.GZipMiddleware",
    ...
)

Or do it before you return a particular response. In your views.py, dec would be the handler for a certain url

from django.middleware.gzip import GZipMiddleware

gzip_middleware = GZipMiddleware()

 def dec(request, *args, **kwargs):
        response = func(request, *args, **kwargs)
        return gzip_middleware.process_response(request, response)
        return dec

NOTE: You should be certain you are not subject to side-channel attacks before using GZip middleware.

Warning

Security researchers recently revealed that when compression techniques (including GZipMiddleware) are used on a website, the site may become exposed to a number of possible attacks. Before using GZipMiddleware on your site, you should consider very carefully whether you are subject to these attacks. If you’re in any doubt about whether you’re affected, you should avoid using GZipMiddleware. For more details, see the the BREACH paper (PDF) and breachattack.com.

Also:

Changed in Django 1.10: In older versions, Django’s CSRF protection mechanism was vulnerable to BREACH attacks when compression was used. This is no longer the case, but you should still take care not to compromise your own secrets this way.

edited Nov 11 '22 at 22:11

Barney Szabolcs

11,846
12
66
91

answered Dec 08 '09 at 03:28

Andres

2,880
4
32
38

3

Please note current security warnings regarding the gzip middleware: https://docs.djangoproject.com/en/1.9/ref/middleware/#module-django.middleware.gzip – Tobias Lorenz Feb 03 '16 at 23:05
When I tried this, I got `No module named middlware.gzip`. Adding `@gzip_page`on top of my view methods worked though. Any idea what the problem is? Am on Django 1.9 – hd1 Mar 11 '16 at 06:19
2

@hd1 you should use strings to specify a middleware class, not the class/module itself. So instead of `MIDDLEWARE_CLASSES = [django.middleware.gzip.GZipMiddleware,]` you should do: `MIDDLEWARE_CLASSES = ["django.middleware.gzip.GZipMiddleware",]` – nik_m Jan 22 '17 at 18:42
As of 1.10, Django includes methods to mitigate the BREACH attack, resolving the security issue Tobias highlighted above. https://docs.djangoproject.com/en/1.11/ref/middleware/ – evan_b Sep 07 '17 at 06:55
Due to security issues and as remarked above, I'd go with the decorator approach. – Jonatas CD Dec 05 '17 at 14:54
What is the meaning of 2 consecutive return statements in the same method? – Fernando Acosta y Lara Jan 04 '23 at 15:45

score 30 · Answer 2 · answered Feb 11 '13 at 22:00

30

If you're gzipping single page, not for all pages, you can use gzip_page decorator instead of GzipMiddleware.

from django.views.decorators.gzip import gzip_page

@gzip_page
def viewFunc(request):
  return HttpResponse("hello"*100)

Reference here: https://docs.djangoproject.com/en/1.4/topics/http/decorators/#module-django.views.decorators.gzip

answered Feb 11 '13 at 22:00

feifan.overflow

565
5
9

By the way, this is still valid for Django 1.11.x and for a Django problem, a Django solution should be the accepted answer IMHO. – Jonatas CD Dec 05 '17 at 14:52

score 27 · Accepted Answer · answered Jun 05 '09 at 19:57

zlib is a bit too low-level for this purpose. Here's how the GZip middleware itself does it (see compress_string in django.utils.text.py):

import cStringIO, gzip
zbuf = cStringIO.StringIO()
zfile = gzip.GzipFile(mode='wb', compresslevel=6, fileobj=zbuf)
zfile.write(template.render(context).encode('utf-8'))
zfile.close()

compressed_content = zbuf.getvalue()
response = HttpResponse(compressed_content)
response['Content-Encoding'] = 'gzip'
response['Content-Length'] = str(len(compressed_content))
return response

GZip uses zlib, but on its own zlib produces content that's improperly encoded for a browser seeing 'gzip' as the content encoding. Hope that helps!

score 2 · Answer 4 · edited May 23 '17 at 12:02

For the sake of others finding this question and who are using nginx, this SO worked for me:

https://stackoverflow.com/a/41820704/4533488

Basically turning gzip on in the /etc/nginx/nginx.conf file did all the compression handling for me. On the client-side, most modern browsers automatically handle extracting (uncompressing) the data when receiving it - sweet!

Here is the nginx.conf file settings:

    http {

        #... other settings ...#

        ##
        # Gzip Settings
        ##

        gzip on;
        gzip_disable "msie6";

        gzip_vary on;
        gzip_proxied any;
        gzip_comp_level 6;
        gzip_buffers 16 8k;
        gzip_http_version 1.1;
        gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    }

Be aware that this only compresses HTTP 1.1 but not HTTP 1.0, which might or might not be an issue depending on your application and target audience. AFAIK there is no simple solution for that. — roskakori, Mar 18 '22 at 16:57

score 1 · Answer 5 · answered Apr 05 '18 at 06:38

If you need it for a single page and you are using class based views, use this:

gzip_middleware = GZipMiddleware()

class GZipMixin(object):

    def dispatch(self, request, *args, **kwargs):
        response = super(GZipMixin, self).dispatch(request, *args, **kwargs)
        return gzip_middleware.process_response(request, response)

Then in your actual view:

class MyView(GZipMixin, View):
    def get(self, request, *args, **kwargs):
         #return your response

score 1 · Answer 6 · edited Jun 20 '20 at 09:12

If you compress your data with zlib, you have to set Content-Encoding to deflate, not gzip.

rendered_page =  zlib.compress(template.render(context).encode('utf-8'))

response = HttpResponse(rendered_page)
response['Content-Encoding'] = 'deflate'
response['Content-Length'] = len(rendered_page)
return response

Content-Encoding

(...)

deflate

Using the zlib structure (defined in RFC 1950) with the deflate compression algorithm (defined in RFC 1951).

Serving gzipped content from django

6 Answers6

Content-Encoding

Linked