0

I've opened a webpage in Selenium, I have access it its url and I would like to save it to a .url file. The file will save and if you open it in vim it displays all the contents of the HTML page (using the first implementation below). That said, when I try and open a dialogue box appears with the error "The target "" of this Internet Shortcut is not valid." Is there any way of easily approaching this in Python?

Implementation 1:

r = requests.get(url, allow_redirects=True)
open('google.url', 'wb').write(r.content)

Implementation 2:

urllib.request.urlretrieve(url, "google.url")
ben.weller
  • 33
  • 6

1 Answers1

0

You can try something like this

    headers = your headers
    req = urllib.Request(url, None, headers)
    page = urllib.urlopen(req).read()
    soupdata = BeautifulSoup(page, "html.parser")

the modules i have used it bs4 and urllib2 as urllib

good luck :)

laslavinco
  • 344
  • 4
  • 14