concurrently click links in multiple tabs using selenium

Question

I am using the following code to invoke selenium to open multiple tabs and click on a particular link on each tab concurrently.

The input file google-search-terms.adoc contains:

5 Dysfunctions of a Team by Patrick Lencioni
Agile Metrics in Action: How to measure and improve team performance
Agile Testing : A Practical Guide for Testers and Agile Teams
Building Great Software Engineering Teams by Josh Tyler
Building Team Power: How to Unleash the Collaborative Genius of Teams for Increased Engagement, Productivity, and Results, by Thomas Kayser

The code

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import urllib.parse
import time
from multiprocessing import Process

start = time.time()

caps = DesiredCapabilities().CHROME
caps["pageLoadStrategy"] = "eager"   # Do not wait for full page load

browser = webdriver.Chrome(desired_capabilities=caps)

def worker(ii):
    browser.switch_to.window(ii)
    try:
        result = browser.find_elements_by_xpath('//div[@id="rso"]/div/div')[0]
        result.find_element_by_xpath("./div/a").click()
    except:
        print("An exception occurred")


all_procs = []
for x in range(1, len(browser.window_handles)):
    p = Process(target=worker, args=(browser.window_handles[x],))
    all_procs.append(p)
    p.start()

for p in all_procs:
    p.join()

print("Total time taken: ", time.time()-start)

Now it is throwing error

p = Process(target=worker, args=(browser.window_handles[x],))
TypeError: 'NoneType' object is not subscriptable

How can I fix that?

Update

I noticed that if I use the debugger and step over slowly, then there is no error and it loads all the pages and click all the links. I think being in multiple tabs in almost the same time is the main issue. Please let me know your suggestions.

Can you call all the `p.join()` loop OUTSIDE the loop which starts the process (i.e. `p.start()` — Rahul Bharadwaj, Oct 07 '20 at 15:23
You can put all the processes in one global list and call join on all outside. Just give it a try, you can refactor code if it works. — Rahul Bharadwaj, Oct 07 '20 at 15:27
@RahulBharadwaj you actually found a mistake in my code. You are right. But after fixing it, it is throwing other errors. — Ahmad Ismail, Oct 07 '20 at 15:37
@RahulBharadwaj I updated the question with your suggestion. — Ahmad Ismail, Oct 07 '20 at 15:45
Since you are using index just change this line and see if works `for x in range(len(browser.window_handles)-1):` — KunduK, Oct 07 '20 at 16:16
@KunduK I just tried, `TypeError: 'NoneType' object is not subscriptable` error. — Ahmad Ismail, Oct 07 '20 at 16:23
https://stackoverflow.com/a/18150682/1772898 says "If you are wanting to have multiple threads all act on the same driver instance, but different tabs, that is NOT possible." Is it still the case? — Ahmad Ismail, Oct 07 '20 at 16:30

concurrently click links in multiple tabs using selenium

0 Answers0