Should I use Selenium grid for the following scenario?

Question

So basically I have a list of URLs. I want to open each URL using webdriver simultaneously so the task can be achieved in a short span of time (instead of looping through each URL in the list).

Should I use Selenium Grid or is there a simpler way?

My code looks as follows:

import selenium
 from selenium import webdriver
 from selenium.webdriver.common.by import By
 from selenium.common.exceptions import NoSuchElementException
 import time
 from selenium.webdriver.common.keys import Keys

 list = ['www.link1.com', 'www.link2.com','www.link3.com'....]

 for i in list:
driver2 = webdriver.Chrome()
driver2.get(i)
time.sleep(1)
try:
    finallinks = []
    all_links = driver2.find_elements(By.XPATH, "/html/body/main/section[1]/div/section[2]/div/div[1]/div/div/div[1]/div/div/section/main/div[2]/form/div[2]/div/div/a")
    print("HOLAAAAAA")
    for a in all_links:
        if     str(a.get_attribute('href')).startswith("https://something/view") and a.get_attribute(
                'href') not in finallinks:
            finallinks.append(a.get_attribute('href'))
            print(finallinks)
except NoSuchElementException:
    print("Didn't exist")




 

Answer 1

if you want to run multiple instances of your webdriver tests in parallel, you can use Selenium Grid. It enables you to distribute your tests across multiple machines and run them in parallel, which can significantly reduce the time it takes to complete a suite of tests.

However, if you are working on a small scale, you can also use multi-threading or multi-processing to run multiple webdriver instances simultaneously in your code. This approach may be simpler than setting up a Selenium Grid, but it will not scale as well if you need to run tests on many machines or if you need to run tests in different environments.

Answer 2

Selenium Grid

Selenium Grid allows the execution of automation scripts on remote machines by routing commands sent by the client to remote browser instances using the WebDriver. Selenium Grid enables us to run tests in parallel on multiple machines and also allows testing on different browser versions enabling cross platform testing.

---

This usecase

A lot depends on the size of the list of urls.

• Case A: In case of 5-10 urls:

  list = ['www.link1.com', 'www.link2.com','www.link3.com'....'www.link10.com']
  

  I would still go with a single node, just to save the cost of maintaining the Selenium Grid

• Case B: In case of more then 10 urls:

  list = ['www.link1.com', 'www.link2.com','www.link3.com'....'www.link20.com']
  

  It would be adviseable to implement Selenium Grid and distribute the urls among the available Selenium Grid Nodes as follows:

  • Node 1:

    list = ['www.link1.com', 'www.link2.com','www.link3.com'....'www.link10.com']
    

  • Node 2:

    list = ['www.link11.com', 'www.link12.com','www.link13.com'....'www.link20.com']