Passing session from template view to python requests api call

Question

I want to make multiple internal REST API call from my Django TemplateView, using requests library. Now I want to pass the session too from template view to api call. What is the recommended way to do that, keeping performance in mind.

Right now, I'm extracting cookie from the current request object in template view, and passing that to requests.get() or requests.post() call. But problem with that is, I would have to pass request object to my API Client, which I don't want.

This the current wrapper I'm using to route my requests:

def wrap_internal_api_call(request, requests_api, uri, data=None, params=None, cookies=None, is_json=False, files=None):
    headers = {'referer': request.META.get('HTTP_REFERER')}
    logger.debug('Request API: %s calling URL: %s', requests_api, uri)
    logger.debug('Referer header sent with requests: %s', headers['referer'])
    if cookies:
        csrf_token = cookies.get('csrftoken', None)
    else:
        csrf_token = request.COOKIES.get('csrftoken', None)

    if csrf_token:
        headers['X-CSRFToken'] = csrf_token
    if data:
        if is_json:
            return requests_api(uri, json=data, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)
        elif not files:
            return requests_api(uri, data=data, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)
        else:
            return requests_api(uri, data=data, files=files, params=params, cookies=cookies if cookies else request.COOKIES,
                                headers=headers)
    else:
        return requests_api(uri, params=params, cookies=cookies if cookies else request.COOKIES, headers=headers)

Basically I want to get rid of that request parameter (1st param), because then to call it I've to keep passing request object from TemplateViews to internal services. Also, how can I keep persistent connection across multiple calls?

Few questions: How do you receive the data?, what are you trying to accomplish in general? Are these outbounds API calls? Do you need to pass the whole session object? Seems like an unusual use case, but I need to see a bigger picture. Can you please explain what are trying to accomplish. Thanks. — mariodev, Jul 22 '16 at 20:20
@mariodev: I've a Django TemplateView. I want to collect data for putting in context of that view. For that I'll call a REST API, that is again in my application only. But making REST API call will be a new call and thus if I don't pass `cookie`, then `session` there would be different from what it is in TemplateView, which I don't want. Basically I want to keep the authentication valid for internal API calls. — Rohit Jain, Jul 23 '16 at 03:36
Whenever dealing with api requests it's always better to pass tokens for auth (smth like JWT should work in this case). Although I'm still confused why are you using api calls at all? Why not just use internal function or class method pointing to the receiving endpoint? As you claim you only use it internally, so I see no point of having api calls really.. — mariodev, Jul 23 '16 at 05:40
Is there a reason you cannot just call the rest API function from within your view and just pass along the data to your template context? Django operates on a cycle of request, view, template, response. You are trying to add a step here and make it request, view, template, view, response, which was never how the stack was designed to work. — Titus P, Jul 28 '16 at 18:01
@TitusP Isn't this a common scenario to have REST API exposed for our application, which is then consumed on website (TemplateView), or android app, or any other platform? Why would Django have restriction for such a common use case? REST APIs are the way system talk. Calling functions to get work done, would work some time, but not in all scenarios. — Rohit Jain, Jul 28 '16 at 19:05
Yes, its a very common scenario. However, those API calls are not made from within the template, they are made AFTER the HTML and javascript is loaded in the user's browser. So, you would render the template, send it back as the response to the request, and then, after the user gets the response back, their browser can execute the javascript necessary to make the REST API call. The REST API is for external data consumption, not internal. Internal to your own Django app, you should simply load up the data and pass it in as the template context. — Titus P, Jul 28 '16 at 19:56
Assuming that you actually need to do API calls, I think this just boils down to some code refactoring. I posted an answer. — Julian, Jul 29 '16 at 01:37

score 5 · Accepted Answer · edited May 23 '17 at 12:14

REST vs Invoking the view directly

While it's possible for a web app to make a REST API call to itself. That's not what REST is designed for. Consider the following from: https://docs.djangoproject.com/ja/1.9/topics/http/middleware/

As you can see a django request/response cycle has quite a bit of overhead. Add to this the overhead of webserver and wsgi container. At the client side you have the overhead associated with the requests library, but hang on a sec, the client also happens to be the same web app so it become s part of the web app's overhead too. And there is the problem of peristence (which I will come to shortly).

Last but not least, if you have a DNS round robin setup your request may actually go out on the wire before coming back to the same server. There is a better way, to invoke the view directly.

To invoke another view without the rest API call is really easy

 other_app.other_view(request, **kwargs)

This has been discussed a few times here at links such as Django Call Class based view from another class based view and Can I call a view from within another view? so I will not elaborate.

Persistent requests

Persistent http requests (talking about python requests rather than django.http.request.HttpRequest) are managed through session objects (again not to be confused with django sessions). Avoiding confusion is really difficult:

The Session object allows you to persist certain parameters across requests. It also persists cookies across all requests made from the Session instance, and will use urllib3's connection pooling. So if you're making several requests to the same host, the underlying TCP connection will be reused, which can result in a significant performance increase

Different hits to your django view will probably be from different users so you don't want to same cookie reused for the internal REST call. The other problem is that the python session object cannot be persisted between two different hit to the django view. Sockets cannot generally be serialized, a requirement for chucking them into memcached or redis.

If you still want to persist with internal REST

I think @julian 's answer shows how to avoid passing the django request instance as a parameter.

So I was trying the direct view calling approach, and was wondering how to pass query parameters to the views? — Rohit Jain, Aug 12 '16 at 13:10
You are passing the whole request object. So the second view can grab it from request.GET — e4c5, Aug 12 '16 at 13:21
Cool. Actually I wanted to send different query parameter. So I modified the `request.GET` object. But the `as_view()` returns a `TemplateResponse` it seems. How do I get the json out of it? With API call, I just did `response.json()` to get. Now when I do `response.render().content`, it's template response. — Rohit Jain, Aug 12 '16 at 13:24
If the view is returning json, json_loads(response.rendered_content) — e4c5, Aug 12 '16 at 13:48

score 1 · Answer 2 · answered Jul 29 '16 at 01:34

If you want to avoid passing the request to wrap_internal_api_call, all you need to do is do a bit more work on the end of the TemplateView where you call the api wrapper. Note that your original wrapper is doing a lot of cookies if cookies else request.COOKIES. You can factor that out to the calling site. Rewrite your api wrapper as follows:

def wrap_internal_api_call(referer, requests_api, uri, data=None, params=None, cookies, is_json=False, files=None):
    headers = {'referer': referer}
    logger.debug('Request API: %s calling URL: %s', requests_api, uri)
    logger.debug('Referer header sent with requests: %s', referer)
    csrf_token = cookies.get('csrftoken', None)

    if csrf_token:
        headers['X-CSRFToken'] = csrf_token
    if data:
        if is_json:
            return requests_api(uri, json=data, params=params, cookies=cookies, headers=headers)
        elif not files:
            return requests_api(uri, data=data, params=params, cookies=cookies, headers=headers)
        else:
            return requests_api(uri, data=data, files=files, params=params, cookies=cookies, headers=headers)
    else:
        return requests_api(uri, params=params, cookies=cookies, headers=headers)

Now, at the place of invocation, instead of

wrap_internal_api_call(request, requests_api, uri, data, params, cookies, is_json, files)

do:

cookies_param = cookies or request.COOKIES
referer_param = request.META.get['HTTP_REFERER']
wrap_internal_api_call(referer_param, requests_api, uri, data, params, cookies_param, is_json, files)

Now you are not passing the request object to the wrapper anymore. This saves a little bit of time because you don't test cookies over and over, but otherwise it doesn't make a difference for performance. In fact, you could achieve the same slight performance gain just by doing the cookies or request.COOKIES once inside the api wrapper.

Networking is always the tightest bottleneck in any application. So if these internal APIs are on the same machine as your TemplateView, your best bet for performance is to avoid doing an API call.

score -1 · Answer 3 · answered Jul 29 '16 at 16:20

Basically I want to get rid of that request parameter (1st param), because then to call it I've to keep passing request object from TemplateViews to internal services.

To pass function args without explicitly passing them into function calls you can use decorators to wrap your functions and automatically inject your arguments. Using this with a global variable and some django middleware for registering the request before it gets to your view will solve your problem. See below for an abstracted and simplified version of what I mean.

request_decorators.py

REQUEST = None


def request_extractor(func):

    def extractor(cls, request, *args, **kwargs):
        global REQUEST
        REQUEST = request # this part registers request arg to global
        return func(cls, request, *args, **kwargs) 

    return extractor


def request_injector(func):

    def injector(*args, **kwargs):
        global REQUEST
        request = REQUEST
        if len(args) > 0 and callable(args[0]): # to make it work with class methods
            return func(args[0], request, args[1:], **kwargs) # class method
        return func(request, *args, **kwargs) # function

    return injector

extract_request_middleware.py

See the django docs for info on setting up middleware

from request_decorators import request_extractor

class ExtractRequest:

    @request_extractor
    def process_request(self, request):
        return None

internal_function.py

from request_decorators import request_injector

@request_injector
def internal_function(request):
    return request

your_view.py

from internal_function import internal_function

def view_with_request(request):
    return internal_function() # here we don't need to pass in the request arg.

def run_test():

    request = "a request!"
    ExtractRequest().process_request(request)
    response = view_with_request(request)
    return response


if __name__ == '__main__':

    assert run_test() == "a request!"

Either, it is entirely unclear how your answer addresses the question, or your answer simply misses the point. To me, it looks like a general trick for passing a parameter covertly, i.e. without it being visible. Then again, this doesn't even seem to work in your example code. In any case, making the parameter "invisible" is not what @RohitJain asked for. — Julian, Jul 29 '16 at 23:38