0

I am trying to read in a url in python 3 however when I tried it did not completely red in the URL Here is my code

my_url="https://www.newegg.ca/Video-Cards-Video-Devices/Category/ID-38?Tpk=graphics%20cards"

Uclient=uReq(my_url)

page_html=Uclient.read()
Andrej Kesely
  • 168,389
  • 15
  • 48
  • 91
  • 1
    Using `requests` the page loads ok. – Andrej Kesely Jul 19 '19 at 17:27
  • Andrej Kesley I did use request –  Jul 19 '19 at 17:32
  • @pelumi I believe Andrej is referring to the [`requests`](https://2.python-requests.org/en/master/) package. – Alex Jul 19 '19 at 17:42
  • I already used the requests function on my url I just called it uReq I am trying to read the url in using the( . read) method however it is giving an incomplete read –  Jul 20 '19 at 18:47

1 Answers1

0

Have you tried importing it using requests? As you do not show you direct imports I am assuming you are using urllib.request. The code below should provide you the entire html text available before any javascript is loaded (if the case)

import requests

my_url="https://www.newegg.ca/Video-Cards-Video-Devices/Category/ID-38?Tpk=graphics%20cards"
r = requests.get(my_url)
print (r.text)
realr
  • 3,652
  • 6
  • 23
  • 34
  • I already used the requests function on my url I just called it uReq I am trying to read the url in using the( . read) method however it is giving an incomplete read –  Jul 20 '19 at 18:46
  • Hi @pelumi, a few things: (1) package requests does not seem to have a read() method (only a raw.read()). You are probably talking about `urllib.request` which is not the same as package `requests`, as pointed out by other users (see this thread: https://stackoverflow.com/questions/2018026/what-are-the-differences-between-the-urllib-urllib2-and-requests-module) . (2) With that said, even using `urllib.request.urlopen(my_url).read()`, the result is still the same as in the page source code, with all html and javascript text parsed. Were you looking for anything that perhaps is loaded using js? – realr Jul 20 '19 at 22:25