Multiple chrome processes after chromedriver.quit()

Question

I am running a Django service that fires up a chromedriver for selenium and scrapes a website for data. The Django service is called by another Java service through HTTP.

Here is the code:

views.py

path_to_chromedriver = '/path/to/chromedriver' 
browser = webdriver.Chrome(executable_path = path_to_chromedriver)
try:
    response = get_data(browser)
except Exception as e:
    print str(e)
finally:
    browser.close()
    browser.quit()

scraper.py

get_data(browser)
    try:
        .
        .
        .
        for i in range(1,6):
            try:
                .
                .
                .
             return "success data"
             except NoSuchElementException:
                 browser.back()
         raise Exception("No results found")
    except Exception as e:
         print str(e)
         raise

The problem is that after the java service has finished making all the calls and the whole process is complete, there are between 25 - 50 chrome processes orphaned in RAM occupying over 1 GB. Is there anything wrong I'm doing here?

What is the `driver` version? I know old versions had this problem. — Guy, Jan 25 '17 at 12:14
Hi, Im using ChromeDriver 2.27, so I dont think that's the problem. You have any idea about stop_client() ? — Anirudh, Jan 25 '17 at 12:29

score 5 · Answer 1 · answered Jan 30 '17 at 16:05

5

That's an old issue. What works for me, although dirty, is to add a sleep before quitting:

time.sleep(5)
browser.quit()

answered Jan 30 '17 at 16:05

Ivan Chaer

6,980
1
38
48

let me give this a shot – Anirudh Jan 30 '17 at 18:30
@Anirudh no luck? – Ivan Chaer Jan 31 '17 at 13:31
let's continue on the chat? http://chat.stackoverflow.com/rooms/info/135757/41744376 – Ivan Chaer Feb 15 '17 at 11:00
turns out the issue was that the Java service was making too many requests to the django service. Ram was simply getting clogged. – Anirudh Feb 20 '17 at 14:33
Cool! Glad you figured it out! – Ivan Chaer Feb 21 '17 at 09:03
@IvanChaer can you please help with the reason behind the solution you've suggested? – Anshul Tiwari Aug 09 '21 at 18:33
Hello @AnshulTiwari. We had this issue 4 years and a half ago, I can't recall. – Ivan Chaer Aug 11 '21 at 16:14

score 0 · Answer 2 · answered Jan 25 '17 at 15:59

0

Do not close before quit and surround code by try/except.

try:
    browser.quit()
except WebDriverException:
    pass

answered Jan 25 '17 at 15:59

srjjio

930
1
8
16

score 0 · Answer 3 · edited May 23 '17 at 12:09

0

Want to expand the answer given by @Serge B.

It's indeed better to use browser.quit() over browser.close() because the last one may close only your current window when the first one will close all windows (and all processes).

Still I don't think you'll solve your problem by doing this.

I had the same issue - a driver is not closed when it's run by a Java process. I encountered it while running my tests with TeamCity. Please try to run your code without the Java service you use and make sure it's the problem.

If it's the problem then I suggest to kill all the processes by Python methods. It's the solution I used and it worked for me. Unfortunately I used C# at that point and the next code helped me https://stackoverflow.com/a/35692319/4019586 but it's not what you can use now.

I don't know how to do it on Python but this may be a solution for you. I think with Python this will work https://stackoverflow.com/a/6278951/4019586.

So the positive side is that you do all correct!) It's just a bug in Selenium.

edited May 23 '17 at 12:09

Community

1
1

answered Jan 25 '17 at 17:08

Denis Koreyba

3,144
1
31
49

the java service simply makes a http call to the django service and times out in 30 seconds. I dont think the Java service is causing the problem. I will try calling only quit but I REALLY doubt that's the issue. – Anirudh Jan 26 '17 at 09:56
Try to call quit running your tests without the java service. – Denis Koreyba Jan 26 '17 at 11:35
@Anirudh what if you run not all your tests but only 1, are the processes closed in this case? – Denis Koreyba Jan 27 '17 at 12:32
THen the process closes – Anirudh Feb 10 '17 at 11:44

Multiple chrome processes after chromedriver.quit()

3 Answers3