Django StaticLiveServerTestCase sometimes ends with: An existing connection was forcibly closed by the remote host

Question

In an Django application that I freshly created, where I changed it to use PostgreSQL and I created one app, I have the following test:

from django.contrib.auth.models import User
from django.contrib.staticfiles.testing import StaticLiveServerTestCase
from selenium import webdriver
from selenium.webdriver.common.keys import Keys


class TestWebBrowser(StaticLiveServerTestCase):
    @classmethod
    def setUpClass(cls):
        super().setUpClass()
        cls.webdriver = webdriver.Chrome()
        cls.webdriver.implicitly_wait(10)

    @classmethod
    def tearDownClass(cls):
        cls.webdriver.close()
        cls.webdriver.quit()
        super().tearDownClass()

    def setUp(self):
        self.admin = User.objects.create_superuser(username="username", password="password",
                                                   email="example@example.com")

    def test_log_in(self):
        self.webdriver.get(f"{self.live_server_url}/admin")
        self.webdriver.find_element_by_id("id_username").send_keys("username")
        self.webdriver.find_element_by_id("id_password").send_keys("password")
        self.webdriver.find_element_by_id("id_password").send_keys(Keys.RETURN)
        self.webdriver.find_element_by_link_text("Users").click()

The test always runs, Chrome starts, does what the test say, but at the end, sometimes it throws this error:

Exception happened during processing of request from ('127.0.0.1', 55283)
Traceback (most recent call last):
  File "C:\Users\pupeno\scoop\apps\python\current\lib\socketserver.py", line 647, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Users\pupeno\scoop\apps\python\current\lib\socketserver.py", line 357, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Users\pupeno\scoop\apps\python\current\lib\socketserver.py", line 717, in __init__
    self.handle()
  File "C:\Users\pupeno\Temporary\untitled\venv\lib\site-packages\django\core\servers\basehttp.py", line 139, in handle
    self.raw_requestline = self.rfile.readline(65537)
  File "C:\Users\pupeno\scoop\apps\python\current\lib\socket.py", line 589, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

All tests pass. I just get that to STDERR. Any ideas why? Am I missing some tearing-down?

If I change tearDownClass to:

@classmethod
def tearDownClass(cls):
    cls.webdriver.quit()

I get the same error with the same frequency (as far as I can observe, without having measured it).

I'm running:

Django==2.1.2
selenium==3.141.0

and

> chromedriver.exe --version
ChromeDriver 2.43.600210 (68dcf5eebde37173d4027fa8635e332711d2874a)

and

Google Chrome Version 70.0.3538.102 (Official Build) (64-bit)

The full output of running the tests look like this:

Testing started at 15:33 ...
C:\Users\pupeno\Temporary\untitled\venv\Scripts\python.exe "C:\Program Files\JetBrains\PyCharm 2018.2.4\helpers\pycharm\django_test_manage.py" test foo.tests.TestImportCRMData C:\Users\pupeno\Temporary\untitled
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 49825)
Traceback (most recent call last):
  File "C:\Users\pupeno\scoop\apps\python\current\lib\socketserver.py", line 647, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Users\pupeno\scoop\apps\python\current\lib\socketserver.py", line 357, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Users\pupeno\scoop\apps\python\current\lib\socketserver.py", line 717, in __init__
    self.handle()
  File "C:\Users\pupeno\Temporary\untitled\venv\lib\site-packages\django\core\servers\basehttp.py", line 139, in handle
    self.raw_requestline = self.rfile.readline(65537)
  File "C:\Users\pupeno\scoop\apps\python\current\lib\socket.py", line 589, in readinto
    return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
----------------------------------------
Destroying test database for alias 'default'...

Process finished with exit code 0

@DebanjanB: I don't think this is a duplicate question. The other one is about not managing to communicate with the chromedriver, while in my case that works. The error happens at the end. — Pablo Fernandez, Nov 15 '18 at 15:20
That's all the stack trace I'm getting. It must be happening on a separate thread. — Pablo Fernandez, Nov 15 '18 at 15:29
Can you help me to identify exactly at which line you are seeing the error? — undetected Selenium, Nov 15 '18 at 15:31
It doesn't happen on any line of my code. My tests all pass, and I just get that to `STDERR` every now and then. — Pablo Fernandez, Nov 15 '18 at 15:32

undetected Selenium · Answer 1 · 2020-01-07T19:13:59.170

This error message...

ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

...implies that the ChromeDriver was unable to communicate with the WebBrowsing Session i.e. Chrome Browser session as it was forcibly closed by the remote host.

Your main issue is the multiple calls in the form of webdriver.close() and webdriver.quit()

While automating through Selenium as per the best practices to terminate a connection you should always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully. Invoking quit() method DELETEs the current browsing session through sending "quit" command with {"flags":["eForceQuit"]} and finally sends the GET request on /shutdown EndPoint. Here is an example below :

1503397488598   webdriver::server   DEBUG   -> DELETE /session/8e457516-3335-4d3b-9140-53fb52aa8b74 
1503397488607   geckodriver::marionette TRACE   -> 37:[0,4,"quit",{"flags":["eForceQuit"]}]
1503397488821   webdriver::server   DEBUG   -> GET /shutdown

So on invoking quit() method the Web Browser session and the WebDriver instance gets killed completely. Hence you don't have to incorporate any additional steps (invoking close()) which will be an overhead by the prevailing standards.

Solution

First of all you remove the following line:

cls.webdriver.close()

The very next line:

cls.webdriver.quit()

Is bound to terminate the connection in the best possible way.

Here you can find a detailed discussion on Selenium : How to stop geckodriver process impacting PC memory, without calling driver.quit()?

Update A

As per the liveservertestcase the tearDownClass() @classmethod isdefined as:

@classmethod
def tearDownClass(cls):
    cls.selenium.quit()
    super().tearDownClass()

As per django/django/test/testcases.py this is called:

@classmethod
def tearDownClass(cls):
    cls._tearDownClassInternal()
    cls._live_server_modified_settings.disable()
    super().tearDownClass()

Update B

Though you mentioned about:

Using ChromeDriver 2.43.600210
Using Google Chrome Version 70.0.3538.102

As per release notes of ChromeDriver v2.43:

Supports Chrome v69-71

So they are compatible but possibly there are multiple installations of Google Chrome installed in your machine and the mismatched version of Google Chrome binary is invoked when the following line is executed:
```
cls.webdriver = webdriver.Chrome()
```

Solution

Ensure that:

Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is updated to current ChromeDriver v79.0.3945.36 level.
Chrome is updated to current Chrome Version 79.0 level. (as per ChromeDriver v79.0 release notes)
Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
(WindowsOS only) Use CCleaner tool to wipe off all the OS chores before and after the execution of your Test Suite.
(LinuxOS only) Free Up and Release the Unused/Cached Memory in Ubuntu/Linux Mint before and after the execution of your Test Suite.
If your base Web Client version is too old, then uninstall it and install a recent GA and released version of Web Client.
Take a System Reboot.
Execute your @Test as non-root user.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.

Reference

You can finda couple of relevant discussions in:

urllib3.exceptions.ProtocolError: ('Connection aborted.', error(10054, 'An existing connection was forcibly closed by the remote host'))

Having just `quit()` and no `close()` results in the same error. I tried all the permutations of these two methods and none worked without an error. — Pablo Fernandez, Nov 15 '18 at 16:27
@pupeno I missed your step `super().tearDownClass()`. Is that necessary? — undetected Selenium, Nov 15 '18 at 16:29
I added it because of the documentation here: https://docs.djangoproject.com/en/2.1/topics/testing/tools/#liveservertestcase. This is the code that's being called: https://github.com/django/django/blob/b3b1d3d45fc066367f4fcacf0b06f72fcd00a9c6/django/test/testcases.py#L1367-L1371 — Pablo Fernandez, Nov 15 '18 at 16:35
As per the reference discussion you need to call `quit()` only once which gracefully clears up the webdriver and web browser. Any additional step will raise an exception. — undetected Selenium, Nov 15 '18 at 16:36
I updated the question with this information. Just calling `quit()` throws the same error. Looking at the backtrace, I think the error is in django's server, not in selenium. — Pablo Fernandez, Nov 15 '18 at 16:42

Django StaticLiveServerTestCase sometimes ends with: An existing connection was forcibly closed by the remote host

1 Answers1

Solution

Update A

Update B

Solution

Reference