How to change the host in next key in a paginated URL in django rest framework?

Question

I have a ModelSerializer in Django Rest Framework with paginated responses. So I have deployed it with gunicorn in a docker container.

gunicorn -c gunicorn_config.py app.wsgi --bind 0.0.0.0:5000

Now the problem is in the paginated responses. The next key is something like.

next: "http://0.0.0.0:5000/admin/users/?page=2&per_page=10"

In my client-side where I am consuming these APIs, I just check the next key and fetch the next response. But since the next key has the host as 0.0.0.0:5000 hence it will cause API call failure. And the purpose is not served for the next key.

So at the moment, my API server is running in a separate docker container. Which is set up via the reverse proxy in nginx.

@AkshatZala port is not the issue, also you may run it in any port you want. Please read about gunicorn. — KdPisda, Jun 17 '20 at 05:19
Does this answer your question? [Gunicorn, no module named 'myproject](https://stackoverflow.com/questions/39460892/gunicorn-no-module-named-myproject) — Akshat Zala, Jun 17 '20 at 05:27

score 5 · Answer 1 · answered Jun 17 '20 at 05:46

The next link in the DRF paginator is generated using the hostname from the request. This is how the hostname is determined in the request:

def _get_raw_host(self):
    """
    Return the HTTP host using the environment or request headers. Skip
    allowed hosts protection, so may return an insecure host.
    """
    # We try three options, in order of decreasing preference.
    if settings.USE_X_FORWARDED_HOST and (
            'HTTP_X_FORWARDED_HOST' in self.META):
        host = self.META['HTTP_X_FORWARDED_HOST']
    elif 'HTTP_HOST' in self.META:
        host = self.META['HTTP_HOST']
    else:
        # Reconstruct the host using the algorithm from PEP 333.
        host = self.META['SERVER_NAME']
        server_port = self.get_port()
        if server_port != ('443' if self.is_secure() else '80'):
            host = '%s:%s' % (host, server_port)
    return host

So, check if the HTTP_X_FORWARDED_HOST header sets the correct hostname you need and if so set USE_X_FORWARDED_HOST to True in your settings. Also make sure that the hostname you need is added to ALLOWED_HOSTS.

You could also override the get_next_link() method in the PageNumberPagination class to supply the needed host/domain name

Well, this did not fix my issue, but after inspiring from overriding thing, I customized my own pagination class. And which fixed this. Sharing the solution in a while. thanks a ton, @Ken4scgolars — KdPisda, Jun 17 '20 at 06:30

score 2 · Accepted Answer · answered Jun 17 '20 at 15:25

So I made a custom pagination class extending PageNumberPagination

from rest_framework.pagination import PageNumberPagination
def replace_query_param(url, key, val):
    """
    Given a URL and a key/val pair, set or replace an item in the query
    parameters of the URL, and return the new URL.
    """
    (scheme, netloc, path, query, fragment) = parse.urlsplit(force_str(url))
    scheme = "https"
    netloc = "api.example.com"
    query_dict = parse.parse_qs(query, keep_blank_values=True)
    query_dict[force_str(key)] = [force_str(val)]
    query = parse.urlencode(sorted(list(query_dict.items())), doseq=True)
    return parse.urlunsplit((scheme, netloc, path, query, fragment))


def remove_query_param(url, key):
    """
    Given a URL and a key/val pair, remove an item in the query
    parameters of the URL, and return the new URL.
    """
    (scheme, netloc, path, query, fragment) = parse.urlsplit(force_str(url))
    scheme = "https"
    netloc = "api.example.com"
    query_dict = parse.parse_qs(query, keep_blank_values=True)
    query_dict.pop(key, None)
    query = parse.urlencode(sorted(list(query_dict.items())), doseq=True)
    return parse.urlunsplit((scheme, netloc, path, query, fragment))

class LargeResultsSetPagination(PageNumberPagination):
    page_size = 1000
    page_size_query_param = 'per_page'
    max_page_size = 1000

    def get_next_link(self):
        if not self.page.has_next():
            return None
        url = self.request.build_absolute_uri()
        page_number = self.page.next_page_number()
        return replace_query_param(url, self.page_query_param, page_number)

    def get_previous_link(self):
        if not self.page.has_previous():
            return None
        url = self.request.build_absolute_uri()
        page_number = self.page.previous_page_number()
        if page_number == 1:
            return remove_query_param(url, self.page_query_param)
        return replace_query_param(url, self.page_query_param, page_number)

Now I am using this pagination class in all my ViewSets

class TestViewSet(viewsets.ModelViewSet):
    permission_classes = [permissions.IsAuthenticated]

    queryset = Test.objects.all().order_by("pk")
    serializer_class = test_serializers.TestSerializer
    pagination_class = LargeResultsSetPagination
    search_fields = ['name', 'description', 'follow_up', 'follow_up_type']
    filter_backends = (filters.SearchFilter,)

And it does the job, the original inspiration https://stackoverflow.com/a/62422235/5884045

score 1 · Answer 3 · answered Feb 16 '22 at 23:12

In our project, the frontend server is on a public domain (domain-a) and proxies /api requests to a backend server on a private domain (domain-b) via another proxy. Our problem appeared similar to yours in that all the URLs generated by DRF used the private hostname (domain-b). However we couldn't just use the X-Forwarded-Host header directly since the value received in this header was not correct.

Instead, we added a custom setting:

USE_X_FORWARDED_HOST = True
HOSTNAME_OVERRIDE = "domain-a"

And then added some custom middleware that put that value into the X-Forwarded-Host header:

from django.conf import settings

class HostnameOverrideMiddleware:

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        request.META["HTTP_X_FORWARDED_HOST"] = settings.HOSTNAME_OVERRIDE
        return self.get_response(request)

Note: the value in HOSTNAME_OVERRIDE will also need to be included in ALLOWED_HOSTS.

How to change the host in next key in a paginated URL in django rest framework?

3 Answers3