How to handle simultaneous requests in Django?

Question

I have a standard function-based view in Django which receives some parameters via POST after the user has clicked a button, computes something and then returns a template with context.

@csrf_exempt
def myview(request, param1, param2):

   if request.method == 'POST':
      return HttpResponseRedirect(reverse("app1:view_name", args=[param1, param2]))

   '''Calculate and database r/w'''

   template = loader.get_template('showData.html')
   return HttpResponse(template.render(context, request))

It works with no problem as long as one request is processed at the time (tested both with runserver and in an Apache server).

However, when I use two devices and click on the button simultaneously in each, both requests are mixed up, run simultaneously, and the website ends up trowing a 500 error, or 404 or sometimes success but cannot GET static files.. (again, tested both with runserver and Apache).

How can I force Django to finish the execution of the current request before starting the next? Or is there a better way to tackle this?

Any light on this will be appreciated. Thanks!

score 1 · Answer 1 · answered Jul 28 '20 at 19:35

1

To coordinate threads within a single server process, use

from threading import RLock

lock = RLock()

and then within myview:

    lock.acquire()
    ...  # get template, render it
    lock.release()

You might start your server with $ uwsgi --processes 1 --threads 2 ...

answered Jul 28 '20 at 19:35

J_H

17,926
4
24
44

I added the RLock to the view in Apache and the process hangs when processing the request (I also added the processes and threads flag in the WSGIDaemonProcess). When implementing it locally with runserver it lets one request finish but hangs the other.. Any idea why this behavior occurs? – marcos Jul 28 '20 at 20:12

score 0 · Answer 2 · answered Jul 28 '20 at 19:31

0

Django web server on local machine is not for production environment. So it processes one request at a time. In production, you need to use WSGI server, like uwsgi. With that your app can be set up to serve more than one request at a time. Check https://docs.djangoproject.com/en/2.1/howto/deployment/wsgi/uwsgi/

answered Jul 28 '20 at 19:31

Olasimbo

965
6
14

Yes, I am using Apache with mod_wsgi, but it shows the same problem. I have set the flag of processes=2 and threads=15. – marcos Jul 28 '20 at 19:54
Are you sure you set it up correctly. These steps could help you confirm what you’ve done. https://www.digitalocean.com/community/tutorials/how-to-serve-django-applications-with-apache-and-mod_wsgi-on-ubuntu-16-04 – Olasimbo Jul 28 '20 at 20:05
Not sure what you mean with "django web server on local machine" . if you mean `manage.py runserver`, then it uses multithreading to handle multiple requests in parallel ( https://stackoverflow.com/questions/64980287/why-can-django-handle-multiple-requests/65866605#65866605 ) – gelonida Jan 25 '21 at 11:51

score 0 · Answer 3 · answered Jul 29 '20 at 14:44

0

I post my solution in case its of any help to other.

Finally I configured Apache with a pre-forking to isolate requests from each other. According to the documentation the pre-forking is advised for sites using non-thread-safe libraries (my case, apparently).

With this fix Apache can handle well simultaneous requests. However I will still be glad to hear if someone else has other suggestions!

answered Jul 29 '20 at 14:44

marcos

93
8

This is probably the fastest solution and probably sufficient for many cases. However it could be a good idea to modify the code such, that it is thread safe meaning using no global variables or protecting each global var access with `Lock`s and calling only thread safe functions. Depending on your code this might be more or less difficult to achieve. This solution might be using a little more RAM than a server, that allows threading though. On the other hand (as Python isn't that efficient with threads) it might be more performant to use N processes compared to using N threads – gelonida Jan 25 '21 at 11:57

gelonida · Answer 4 · 2021-01-25T12:03:35.597

There should be ways to rewrite the code such, that things do not get mixed up. (At least in many cases this is possible)

One of the pre-requirements (if your server uses threading) is to write thread safe code This means not using global variables (which is bad practice anyway) (or protecting them with Locks) and using no calls to functions that aren't thread safe. (or protect them with Locks)

As you don't provide any details we cannot help with this. (this = finding a way to not make the whole request blocking, but keep data integrity)

Otherwise you could use a mutex / Lock, that works across multiple processes.

you could for example try to access a locked file https://pypi.org/project/filelock/ and block until the file is unlocked by the other view.

example code (after pip installing filelock)

from filelock import FileLock
lock = FileLock("my.lock")
with lock:
    if request.method == 'POST':
        return HttpResponseRedirect(reverse("app1:view_name", args=[param1, param2]))

    '''Calculate and database r/w'''

    template = loader.get_template('showData.html')
    return HttpResponse(template.render(context, request))

If you use uwsgi, then you could look at the uwsgi implementation of locks:

https://uwsgi-docs.readthedocs.io/en/latest/Locks.html

Here the example code from the uwsgi documentation:

def use_lock_zero_for_important_things():
    uwsgi.lock() # Implicit parameter 0
    # Critical section
    uwsgi.unlock() # Implicit parameter 0

def use_another_lock():
    uwsgi.lock(1)
    time.sleep(1) # Take that, performance! Ha!
    uwsgi.unlock(1)

How to handle simultaneous requests in Django?

4 Answers4