4

I wonder if the there is a canonical approach to this in Django?

Essentially the way views are structured, they are functions that return a result that is delivered to the client. It can be as simple as this:

from django.http import HttpResponse
 
def my_view(request):
    return HttpResponse('This is my view!')

But let us suppose that when my_view is loaded, there is some meat in it:

from django.http import HttpResponse
 
def my_view(request):
    result = do_stuff()
    return HttpResponse(result)

And imagine that do_stuff has some moderately time-consuming wind down code to run, that is code that the result does not depend on, but that it would like to run all the same (perhaps to release resources, return some stuff to a clean state etc. it matters not, the principle is that do_stuff has some code it wants to run, it's a little slow, and hence running it delays delivery of the result, but the result in no way whatsoever depends upon it, so it would prefer to return the result and then run this code).

In a ridiculous imaginary and nonsensical world, this might resemble:

from django.http import HttpResponse
 
def my_view(request):
    result = do_stuff()
    return HttpResponse(result)
    do_more_stuff()

But of course do_more_stuff never runs in the real world because of the very nature of returning.

Now, I can imagine solutions here that spins do_more_stuff() off onto a background thread or into a child process for completion, allowing my_view to return.

The reason I am asking, though, is twofold:

  1. Regarding the lifespan of such a spun-off thread of process, there is a slight mystery here, that I could laboriously test or study deeply, but others may have a handle on. Is it guaranteed to run to completion (assuming modest run time and not something exceptional like a hang, deadlock or month long run time ..., but maybe 1–5 seconds real time and delaying delivery of the view this long is radically noticeable in the page view lag). The view function is run within a process, and so there are questions regarding the lifecycle of that process and any threads or children it spawns. I sort of imagine it's robust that if Django is running under uwsgi say that views are not handled in ephemeral processes but in processes that linger and continue to handle views and as such are around for the long haul and any reasonable background thread or child process will be able to complete comfortably.

  2. Perhaps there is a canonical Django way to do this. I am curious because it can't be that I'm the first or only person to have struck such a need (i.e. found code in a view that is slowing down delivery but upon which the response does not depend) and that it must be a fairly common thing for developers to encounter and may have a canonical solution I have not found. I've checked middleware options, but it is similarly constrained, i.e. like a view returns a response.

I have a suspicion a solution may reside in the request-finished signal:

https://docs.djangoproject.com/en/3.0/ref/signals/#request-finished

and that a function hooked to that signal runs after the response is delivered and does not delay delivery of the response. But again, short of testing it I don't know, and so am happy for input (while I run off and test it).

Bernd Wechner
  • 1,854
  • 1
  • 15
  • 32
  • you'll want to use task queue like celery https://docs.celeryproject.org/en/stable/django/first-steps-with-django.html – iklinac May 24 '20 at 11:34
  • 1
    If the community votes, the community votes and three votes apparently closed this question as opinion based and not possible to answer with facts. I can only suggest that asking a community of Django gurus for a canonical approach is arguably in either camp (answerable by fact, there is or isn't a canonical approach or answerable with opinion, because there isn't one, here's what I'd suggest doing) . The thing about a canonical approach is that it appears in the canon (of Django documentation). I haven't found one. But suspected there may be one buried somewhere. – Bernd Wechner May 25 '20 at 23:35

1 Answers1

1

The signal may do the trick for you, although you can't access the details of the request (the sender arg is just the class, not the request).

You can also extend the response class: Execute code in Django after response has been sent to the client

finally, if you do want to run things in a separate process, consider a distributed task queue like celery (https://docs.celeryproject.org/en/stable/)

Jonah
  • 727
  • 5
  • 12
  • Now there's a deep irony in that I am already using Celery extensively and the code in question I actually specifically want to run client side (of the Celery client-worker relationship) not in a worker. Have been looking at the signal. Alas it provides no clue of what the request was and is generic and fires after every request, so there's not enough information in its context to decide if the view in question was run and its wind down code should run. – Bernd Wechner May 24 '20 at 11:51
  • What's your motivation for not doing the work in a worker? – Jonah May 24 '20 at 11:53
  • Not really relevant to the question, as I'm curious regardless, but the motivation is multifaceted (I do a lot of stuff in workers right now) and relates to wanting to keep this code independent free of the vagaries of workers (availability queuing, context - as in, easy access to request data form data and more). I mean it's not that I **can't** do it in a worker (Celery rocks) but that I am very keen to keep this code client side and be that right or wrong curious what the options are. It could even be that using a worker is the best approach. But I remain curious of other approaches. – Bernd Wechner May 24 '20 at 12:00
  • fair enough. this looks like a pretty clever option. extend the response type: https://stackoverflow.com/questions/4313508/execute-code-in-django-after-response-has-been-sent-to-the-client – Jonah May 24 '20 at 12:04
  • Wow, I looked quite hard for an existing thread and didn't find that one. Good find! – Bernd Wechner May 24 '20 at 12:09
  • @BerndWechner What do you mean by "keep this code client side"? I assume by "this code" you mean your "do_more_stuff()" function. But your example code in the question is server side. – Code-Apprentice May 23 '22 at 01:06
  • 1
    @Code-Apprentice, yes my bad there for lack of clarity. I was thinking celery server and client, not HTTP server and client. The Celery client is in fact the web server - spot on and the language a tad confusing. Thanks for checking in. The Celery server is an abstraction that could be a separate process on the same machine or on a remote machine and so brings with it a good few serialization and communication issues. – Bernd Wechner May 23 '22 at 01:15