3

I've coded the backend of my recently launched mobile application using django 1.9, and the database I've used is MySQL. I'm about to start a large promotional campaign so trying to gear up the backend for the same.

One issue that I'm constantly facing is that when I fire just around 100 threads in a span of 10 seconds via Apache Jmeter, a large number (~60) of them do not get served. Django server logs look like this:

[24/Aug/2016 22:43:01] "GET /my/api/ HTTP/1.0" 200 78547 
[24/Aug/2016 22:43:01] "GET /my/api/ HTTP/1.0" 200 78547
- Broken pipe from ('127.0.0.1', 53582)
- Broken pipe from ('127.0.0.1', 53597) 
[24/Aug/2016 22:43:01] "GET /my/api/ HTTP/1.0" 200 78547
- Broken pipe from ('127.0.0.1', 53551)

What exactly does the broken pipe signify? How do I get around this?

Chetan
  • 1,724
  • 2
  • 14
  • 18
  • 2
    "What more should I do to be ready for a large number of hits?" Close: Too broad. "Broken pipe... what does this mean and how do I fix it?" Answerable, would upvote. Please revise your question to be _one question_. I recommend the latter. ;) – Two-Bit Alchemist Aug 24 '16 at 18:39
  • thought so too. edited. :) – Chetan Aug 24 '16 at 18:41
  • Are you testing jMeter with nginx load balancing in front or are you just hitting the Django debug server? – Two-Bit Alchemist Aug 24 '16 at 19:00
  • I'm using nginx to route requests to my Django debug server. Does that make a difference here? – Chetan Aug 25 '16 at 08:32
  • Yes, a quite large one. The built-in webserver is not meant to be performance ready so there's no point load-testing it. Usually you want nginx (or something like it but it's quite good and scaleable) load-balancing in front of several uwsgi servers to run Django at scale. – Two-Bit Alchemist Aug 25 '16 at 12:23

1 Answers1

0

The usual technique for scaling into make multiple instances (different computers) and then apply static file caching and load balancing with tools like NGinx, Varnish, and HAProxy.

Broken pipes are explained in this SO answer: https://stackoverflow.com/a/30091579/1001643

Community
  • 1
  • 1
Raymond Hettinger
  • 216,523
  • 63
  • 388
  • 485