I had the same problem while scraping data using the Tor network. I got the same exception. Below is the code of the function designed to support the Tor network that caused the problem:
def firefoxdriver(my_url):
"""
Preparing of the Tor browser for the work and adding the headers
to the browser.
"""
# Preparing of the Tor browser for the work.
# for my laptop
# torexe = os.popen(\
# r'C:\Users\Oliver\Desktop\Tor Browser\Browser\firefox.exe')
# for my mainframe
torexe = os.popen(\
r'C:\Users\olive\OneDrive\Pulpit\Tor Browser\Browser\firefox.exe')
# for my laptop
# profile = FirefoxProfile(\
# r'C:\Users\Oliver\Desktop\Tor Browser\Browser\TorBrowser\Data\'+
# 'Browser\profile.default')
# for my mainframe
profile = FirefoxProfile(\
r'C:\Users\olive\OneDrive\Pulpit\Tor Browser\Browser\TorBrowser\Data'+
'\Browser\profile.default')
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9150)
profile.set_preference('network.proxy.socks_remote_dns', False)
profile.update_preferences()
firefox_options = Options()
driver = Firefox(firefox_profile=profile, options=firefox_options)
# # Adding the headers to the browser.
_addingheaders(my_url)
In this code, I only replaced the value False with the value True in the code line of this function as follows:
profile.set_preference('network.proxy.socks_remote_dns', True)
This completely solved the problem.