Selenium Python handeling multiple webdriver at once

Question

Hay, I wane ask how I can handel multiple chrome-browser at once. I tried it with multiple chromedriver.exe, for each browser I wanna open, a own chromedriver.exe

driverPlace2 = "C:/Users/M/PycharmProjects/Test/chromedriver2.exe"
driverPlace3 = "C:/Users/M/PycharmProjects/Test/chromedriver3.exe"
driverPlace4 = "C:/Users/M/PycharmProjects/Test/chromedriver4.exe"
driverPlace5 = "C:/Users/M/PycharmProjects/Test/chromedriver5.exe"
driverPlace6 = "C:/Users/M/PycharmProjects/Test/chromedriver6.exe"
driverPlace7 = "C:/Users/M/PycharmProjects/Test/chromedriver7.exe"
driverPlace8 = "C:/Users/M/PycharmProjects/Test/chromedriver8.exe"

after the 6st one it start to close the second, third ...

it gives nothing out in the console, it just starting to close.

how can i fix it?

thank you very much for reading and helping me

score 0 · Accepted Answer · answered Dec 12 '19 at 22:33

You can use a program with Thread:(the following script, opens 2 chrome-browser, that crawl around the web at the same time)

from selenium import webdriver
from random import shuffle
from concurrent import futures
from urllib.parse import urlparse


def driver_crawl_web(first_page):
    driver = webdriver.Chrome()
    driver.get(first_page)
    while True:
        a_tags = driver.find_elements_by_xpath("//a[@href]")
        shuffle(a_tags)
        for tag in a_tags:
            href = tag.get_attribute('href')
            if check_url(href):
                selected_link = href
                break
        if selected_link == driver.current_url:
            driver.back()
            continue
        driver.get(selected_link)

def check_url(url):
    parse = urlparse(url)
    if "https" in parse.scheme:
        if "youtube" not in parse.netloc:
            return True
    return False

def crawl_with_concurrency(links):
    executor = futures.ThreadPoolExecutor(max_workers=2)
    executions = executor.map(driver_crawl_web, links)


links = ["https://www.google.com/search?q=python",
        "https://www.google.com/search?q=javascript"]

crawl_with_concurrency(links)

thx man so when i understan that right it stays open as long as while True:with something with = like a = 1 stands there right? — IdontLikeNames, Dec 12 '19 at 22:47
Yes, while there is something that is referring to the chrome-browser-driver object, it will stays open — gefgu, Dec 12 '19 at 22:53
i tryed it now with 8 browser (with my method after 7 it close all) now evrything work thx man you really helped me so much im looking for a solution like a month now <3 — IdontLikeNames, Dec 12 '19 at 23:03

Selenium Python handeling multiple webdriver at once

1 Answers1