0

I am running the flathunter script everyday for 14 hours and it opens a chromedriver instance on each execution

https://github.com/flathunters/flathunter/blob/main/flathunter/abstract_crawler.py#L48-L56

After a while, I have a lot of those processes openenter image description here

If that is the case, new instaces of webdriver may fail because of "port already in use" error.

Is it possible to run the webdriver in python in such a way, that it closes the webdriver process when the flathunter.py is exiting (either manually or because of failure)?

Adam
  • 25,960
  • 22
  • 158
  • 247
  • `driver.close()`? – MSH Mar 10 '22 at 20:36
  • @MSH I the script quits because of failure or manual abortion, how would you execute any code? (the script is an infite loop, it never stops gracefully) – Adam Mar 10 '22 at 20:37
  • You should start using `try-catch`s. You can capture any failure/keyboarinterrupt. see: https://stackoverflow.com/a/21144662/2681662 – MSH Mar 10 '22 at 20:42
  • 1
    @MSH I have not written it by myself, its a community project. And they use trycatch, but there are still unhandled errors, also on some errors you want to kill your script, or maybe you restart your machine etc – Adam Mar 10 '22 at 20:44

2 Answers2

2

You can wrap all your Selenium code with try-except-finally block. In the finally block you can put a driver.close() method so that it will close the driver process in case of uncaught exception / failure.
The disadvantage of this approach is that this may cause difficulties with catching the failures with reports listeners.

Prophet
  • 32,350
  • 22
  • 54
  • 79
1

Just before flathunter.py exists kill all of the ChromeDriver processes through proc.kill() as follows:

import os
import psutil

PROCNAME = "chromedriver" # or geckodriver or IEDriverServer
for proc in psutil.process_iter():
    # check whether the process name matches
    if proc.name() == PROCNAME:
        proc.kill()
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • flathunter script will never exit byits own, its an infinite for loop. It only exits on failure or if human kills the process. So I guess I would need to call this code just before L48 that usually starts the webdriver? – Adam Mar 10 '22 at 20:41