0

I'm trying to scrape a website that contains judicial information of my country (Colombia). I have a python script that uses Selenium to open the website and later insert a process number:

pathDriver = 'yourpathdriver'
driver = webdriver.Chrome(executable_path=pathDriver)
url   = 'https://consultaprocesos.ramajudicial.gov.co/Procesos/NumeroRadicacion'
driver.get(url)

However the script only works the first time is executed, in later executions I get this error:

selenium.common.exceptions.WebDriverException: Message: unknown error: net::ERR_CONNECTION_CLOSED

enter image description here

I have to wait about 30 minutes to try the script again, but the result is the same, only works the first time.

I've tried to open the browser with the --incognito flag but this doesn't work. Also, I've tried to find a way to send request headers with Selenium but it seems this feature is not supported.

I am using Windows 10 and ChromeDriver.

Is there any Selenium tip to overcome this issue?

Thanks

K. B.
  • 3,342
  • 3
  • 19
  • 32
David López
  • 500
  • 5
  • 21

1 Answers1

0

When I have seen this error, it was a network issue (site not accessible from internal company network). To confirm or exclude this, try to run the tests from a computer outside your company, for example, your home computer. Here are more suggestions, but some of them are advanced (dangerous) and you should execute them only if you know what you are doing.
Additionally, the site is loaded on my computer for more than 20 seconds and in the console, I see the error:

 GET https://consultaprocesos.ramajudicial.gov.co/js/chunk-3b114a7f.921eecf3.js net::ERR_CONNECTION_TIMED_OUT

However, this does not seem to cause the observed behavior.

Another possible reason could be an outdated browser/WebDriver or incorrect disposal (quit()) of the driver. If the issue is not reproduced manually (opening the site without Selenium), you can try with another WebDriver. You are using Chrome, so try with Firefox.

K. B.
  • 3,342
  • 3
  • 19
  • 32