Selenium node does not kill processes after webrdiver sessions

Question

I run `Flask + Celery + Selenium Hub + Selenium Node.

I run a lot of tasks that use Remote webdriver:

class Task:
    def setup_driver(self):
        chrome_options = webdriver.ChromeOptions()
        chrome_options.add_argument('--headless')
        chrome_options.add_argument('--disable-gpu')
        chrome_options.add_argument('--no-sandbox')
        chrome_options.add_argument("--disable-notifications")
        chrome_options.add_experimental_option('useAutomationExtension', False)
        self.driver = webdriver.Remote(
            command_executor='url',
            options=chrome_options
        )
        self.driver.implicitly_wait(5)

    def teardown_driver(self):
        self.driver.quit()
    
    def run(self):
        self.setup_driver()
        try:
            pass
            # do some logic
        finally:
            self.teardown_driver()

Always I call driver.quit() on the end of task My docker-compose.yaml:

version: "3.3"

services:
  webserver:
    build: .
    ports:
      - 5000:5000
    volumes:
      - ./src:/app
      - ./conf.yaml:/conf.yaml
      - ./db:/db
    restart: unless-stopped
    command: 'uwsgi app.ini'

  redis:
    image: redis:latest
    ports:
    - 6379:6379

  celery:
    build: .
    volumes:
      - ./db:/db
    command: celery -A app.celery_app worker --loglevel=INFO

  celerybeat:
    build: .
    command: celery -A app.celery_app beat --loglevel=INFO --scheduler common.scheduler.TaskLockScheduler

  chrome-node-1:
    image: selenium/node-chrome:91.0
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_NODE_SESSION_TIMEOUT=300
      - SE_NODE_MAX_SESSIONS=10
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
    ports:
      - "6900:5900"

  chrome-node-2:
    image: selenium/node-chrome:91.0
    volumes:
      - /dev/shm:/dev/shm
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_NODE_SESSION_TIMEOUT=300
      - SE_NODE_MAX_SESSIONS=10
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
    ports:
      - "6901:5900"

  selenium-hub:
    image: selenium/hub:4.0.0
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

After few hours htop show me that information: A lot of node and hub processes.

So my question: why hub and node does not kill this processes? A use driver.quit() after each session.

score 0 · Answer 1 · answered Jul 11 '21 at 18:14

I see that finally block may not be executed in several cases.
So, to say more specifically we need to see the entire code inside your try block, but generally it is possible that finally block will be skipped.
If so, this may describe skipping the teardown_driver() method where self.driver.quit() is involved.

Selenium node does not kill processes after webrdiver sessions

1 Answers1