Why does django ignore HTTP_X_FORWARDED_PROTO from the wire but not in tests?

Question

Why does django ignore the HTTP_X_FORWARDED_PROTO if it comes through the wire?

I added to the settings.xml the following config:

# make sure we know when we are secure when we are behind a proxy
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

I made a test to test that if

def testHttpSupport(self):
    url = reverse('configuration-list')
    response = self.client.get(url, HTTP_X_FORWARDED_PROTO='https')
    cfg = response.data[0]
    cfg_url = cfg['url']
    self.assertTrue(cfg_url.startswith('https'))

this works fine. The url of the return object starts with https.

however if I try :

curl -v -H 'HTTP_X_FORWARDED_PROTO: https' http://localhost:8000/api/users/
...
> GET /api/users/ HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/7.51.0
> Accept: */*
> HTTP_X_FORWARDED_PROTO: https
>
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Mon, 03 Jul 2017 16:22:04 GMT
< Server: WSGIServer/0.2 CPython/3.6.1
< Content-Type: application/json
< Allow: GET, POST, OPTIONS
< Vary: Accept, Cookie
< X-Frame-Options: SAMEORIGIN
< Content-Length: 197 
<
* Curl_http_done: called premature == 0
* Closing connection 0
[{"url":"http://localhost:8000/api/users/1/",...

How come it does not return 'https://' based urls like in my unit-test?

Answer 1

The issue is the header name. When accessing Django through a WSGI server, you should use the X-Forwarded-Proto header instead of the HTTP_X_FORWARDED_PROTO:

curl -v -H 'X-Forwarded-Proto: https' http://localhost:8000/api/users/

The WSGI protocol states that the relevant CGI specifications must be followed, which say:

Meta-variables with names beginning with 'HTTP_' contain values read from the client request header fields, if the protocol used is HTTP. The HTTP header field name is converted to upper case, has all occurrences of "-" replaced with "_" and has 'HTTP_' prepended to give the meta-variable name.

(source)

So whenever you are using a WSGI server, the X-Forwarded-Proto header is automatically converted to HTTP_X_FORWARDED_PROTO before it is passed in to Django. When you pass in the HTTP_X_FORWARDED_PROTO header instead, HTTP_ must still be prepended according to the specification. Thus, you end up with a header named HTTP_HTTP_X_FORWARDED_PROTO in Django.

self.client is not a WSGI server, and values passed in through the kwargs are inserted directly into the WSGI environment, without any processing. So in that case you have to do the conversion yourself and actually use the HTTP_X_FORWARDED_PROTO key:

CGI specification

The headers sent via **extra should follow CGI specification. For example, emulating a different “Host” header as sent in the HTTP request from the browser to the server should be passed as HTTP_HOST.

(source)