-1

I have a set of url's in an xml file. I'm planning to write a python program which will read each url and navigate to that web page and wait for a fixed time (eg. 30 sec) and then navigate to next url. The program will have to run for a very long time (say for 2 months and more).

from selenium import webdriver
driver = webdriver.Ie("e:\\IEDriver\\IEDriverServer.exe")
for url in url_list:
    driver.get(url)

So using the above, are there any chances of program getting crashed while running for such prolonged period due to some reason unknown to me? Also any checks I need to do within the program like, memory usage, space usage etc. that will prevent the program from crashing.

If there are better ways to do this, please help me. Thanks in advance

Pranav
  • 115
  • 1
  • 11

2 Answers2

1

There's no reason to leave the program running for multiple months. That's inviting problems. It would be better if you ran your URL loop only once and quit. Package that script up and then call it using Windows Task Scheduler or an equivalent. You can set it to run every hour or whatever time period you want. If you have a power outage, the computer reboots, etc., it will resume execution without manual intervention.

See more about Windows Task Scheduler.

JeffC
  • 22,180
  • 5
  • 32
  • 55
0

As per your code block I don't see any issue in navigating to that web page and wait for a fixed time (eg. 30 sec) and then navigate to next url. But the main factor is what would you like the webdriver instance to do for the specified time. The only way looks viable to me is inducing sleep or keep the browser busy doing nothing where there will be no chances of your program getting crashed.

So the simplest solution will be to induce sleep as follows :

time.sleep(30)

As an enhancement you can configure a pre-defined time for each iteration to happen following the discussion python and selenium webdriver wait until time was equal to specific

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352