Compress JSON payload in Django for both request from client and response from server (REST API).

Question

I am wondering if there's a way to compress the JSON payload I have to send every time with many repetitive field names. While this question is only about compressing the response, I would like to know how to compress the JSON payload from client (may be a mobile app) as well. Also, I want to know how do I convert the compressed JSON file back to its original structure both on client and server side.

A detailed answer with the steps will be appreciated.

I am using djangorestframework==2.4.4 with Django==1.7.7 for the APIs.

Answer 1

From the client-side different tools may have done it differently. A simple implementation for a python-requests based client is done in this post.

But as for the decompression, I think it is better to be done at webserver level, just the same as what you did for response compression. It seems that there is no built-in configuration for Nginx but somebody has done sort of Lua to do the decompression before passing the request to the upstream.

Another - may be less efficient - solution would be to do the decompression in the very first Django middleware like the following:

import gzip


class SimpleMiddleware:
    def __init__(self, get_response):
       self.get_response = get_response

    def __call__(self, request):

        # check the headers if you are writing a versatile API
        # decompress the request body
        request._body = gzip.decompress(request.body)
        # fill the request stream in case you're using something like django-rest-framework
        request._stream = BytesIO(request.body)

        response = self.get_response(request)

        return response

Also, you have to configure your middleware as the very first middleware:

# in Django settings

MIDDLEWARE = [
    'python path to our new custom middleware',
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

Here are the references:

1. Stackoverflow post on how to send gzipped reqeusts,
2. Python 3 gzip documentation,
3. Server fault thread on request body decompression,
4. Django middleware reference.