2

I am working on a Django app on Ubuntu 16.04, and recently noticed that functional tests using selenium in the test suite for my Django app will sometimes fail (timeout). These are tests that previously passed, and no code has been changed. When this happens, it is seemingly random which tests will fail; on one run test A will fail, on the next run test B will fail and A will pass. However, it seems like what usually fails is calls to outside web services (Google Maps API calls), or loading external libraries (loading Font Awesome or Bootstrap through a CDN) - selenium will timeout waiting for these resources to load.

When this happens, running the site locally with the development server (./manage.py runserver) will usually be slower than usual (i.e., some pages will load much slower than usual, or will fail to load altogether).

My suspicion is that some sort of network traffic on my computer is blocking or slowing down Django's development and test server, like another process is using/trying to use the same port or something of that nature.

At one point when this issue was particularly bad, I realized I was trying to upload a ~1GB file to dropbox on the same machine. When I stopped this process my test suite and development server both started to run without issue. Another time, I was uploading music files through Google Music Manager, and when I quit music manager, the tests and dev server went back to normal. However, I am experiencing the same issue at the moment, and I am not currently doing any major file downloading or uploading (that I am aware of).

So, assuming that these are not coincidences, is it the case that any kind of heavy network traffic will cause issues with the Django development server and test server, or is this caused by certain processes on my machine trying to use the same port/resources/something as the Django server? How can I diagnose/solve such a problem?

I have searched SO and elsewhere for insight into the issue, but I am having trouble finding a solution, partially I think because I am not really sure what the problem is or how to articulate it in a way that the search engine understands.

I have tried using the netstat tool, with the suspicion being that some other process was running on 127.0.0.1:8000 (for the dev server run with runserver) and 127.0.0.1:8081 (which is where the test server seems to run), but I will admit that I am not really even sure what I should be looking for there, or if I should be looking there at all.

Please forgive my ignorance; I am new to networking issues (if that is what I am experiencing) and I am aware that I am probably talking about this in a way that will make veterans' skin crawl. I apologize in advance if this question is too vague. If you must downvote, please let me know additional information I can provide to make the question clearer/more useful.

In the time I have spent preparing this post, the problem has once again cleared up on its own, and all my tests pass, but I know it will come back, so any advice would be appreciated. Thank you!

elethan
  • 16,408
  • 8
  • 64
  • 87
  • 2
    `sudo netstat -putan` shall show you any processes listening on the same port (look if you have `0.0.0.0:8000` vs `127.0.0.1:8000`). Anyway, your issue seems to be that sometimes your network latency grows too much. This tends to happen when a high number of outgoing connections is started and your upload bandwidth is reduced (ie while uploading big files) and specially when your connection is asymmetric. Is your Internet connection asymmetric (larger download than upload)?. If so, you need to ensure your bandwidth is enough, either by stopping services or by using traffic shaping. – jjmontes Aug 24 '16 at 16:50
  • Thank you, this information is very helpful. Yes, it appears that I do have an asymmetric connection (4.93 Mbps down, 0.79 up at last test). Next time it happens I will try to compare network activity to that when I am not experiencing the issue. – elethan Aug 24 '16 at 17:12
  • Which OS is this? – o9000 Aug 24 '16 at 21:07
  • @o9000 yeah, I forgot to mention that in my post - kind of important! I am on Ubuntu 16.04. – elethan Aug 24 '16 at 21:39

1 Answers1

1

One possible issue is that the Django server, in development mode, will store all database requests.

It is also important to remember that when running with DEBUG turned on, Django will remember every SQL query it executes. This is useful when you’re debugging, but it’ll rapidly consume memory on a production server.

Does the same behavior happen if you turn DEBUG = False in settings? Does restarting the server solve it?

Kevin London
  • 4,628
  • 1
  • 21
  • 29
  • Restarting the server does not solve it; I have tried that. However, I have not tested whether changing `DEBUG` to `False` helps the issue. I will try this for sure next time it starts to happen to see what the effect is. Thanks for the suggestion! – elethan Aug 24 '16 at 16:53