Selenium Threads: How to open the same browser in multiple threads? for the purposes of each browser using a unique proxy

Question

import threading

def rand_function1():
  #random actions

def rand_function2():
  #random actions

def main()
  rand_function1
  rand_function2
  return


if __name__ == '__main__':
    url_list = "https://www.rand_urls.com/"
    driver = webdriver.Firefox()
    for t in range(10):
        t = threading.Thread(target=main)
        t.start()

I have this simple program that I am trying to open urls using 10 Firefox web drivers. However, all it does it use one browser and continue to cycle though urls thought that individual browser. I will be using a unique proxies for each browser so opening tabs wont be an option.

How do I get n threads to run the main function individually using its own Firefox web driver?

There is a possibility to open 10 links each in separate `Firefox` tab. Do you search for this or you really want to manage 10 browser sessions within same script? — Andersson, Jan 18 '17 at 10:02
Well if you really want to do that, create `driver` in `main()` so that each thread uses a different instance — Shane, Jan 18 '17 at 10:19

score 6 · Answer 1 · answered May 25 '17 at 14:29

According to this and this previous question, selenium is not thread safe.

You should create drivers inside your main, so that every thread has its own driver.

import threading

def rand_function1():
  #random actions

def rand_function2():
  #random actions

def main()
  # use a different driver for each thread
  driver = webdriver.Firefox()
  rand_function1
  rand_function2
  return


if __name__ == '__main__':
    url_list = "https://www.rand_urls.com/"
    for t in range(10):
        t = threading.Thread(target=main)
        t.start()

Selenium Threads: How to open the same browser in multiple threads? for the purposes of each browser using a unique proxy

1 Answers1