3

I've tried a lot way to read the content or extract background HTML from other website using Python 3.5.

I'm familiar with Using an HTTP PROXY, I develop my project on Windows Server 2008 r2, and it's the policy of the company.

Below is my simple code to be able to request connection to the other website.

 import requests
 r = requests.get("https://stackoverflow.com", 
             proxies={"http": "http://proxy.com:900"})
 print(r.text)

Its work when I change the url to google urls as below

r = requests.get("https://www.google.com", 
             proxies={"http": "http://proxy.com:900"})

When i use url from another website.

Then, I got this error on the output.

  C:\Python35\python.exe C:/Users/mwirzonw/PycharmProjects/untitled/test.py
  Traceback (most recent call last):
  File "C:\Python35\lib\site-packages\urllib3\connection.py", line 141, in 
 _new_conn
 (self.host, self.port), self.timeout, **extra_kw)
 File "C:\Python35\lib\site-packages\urllib3\util\connection.py", line 83, 
 in create_connection
 raise err
 File "C:\Python35\lib\site-packages\urllib3\util\connection.py", line 73, 
 in create_connection
 sock.connect(sa)
 TimeoutError: [WinError 10060] A connection attempt failed because the 
 connected party did not properly respond after a period of time, or 
 established connection failed because connected host has failed to respond

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
 File "C:\Python35\lib\site-packages\urllib3\connectionpool.py", line 600, 
 in urlopen
 chunked=chunked)
 File "C:\Python35\lib\site-packages\urllib3\connectionpool.py", line 345, 
 in _make_request
 self._validate_conn(conn)
 File "C:\Python35\lib\site-packages\urllib3\connectionpool.py", line 844, 
 in 
 _validate_conn
 conn.connect()
 File "C:\Python35\lib\site-packages\urllib3\connection.py", line 284, in 
 connect
 conn = self._new_conn()
 File "C:\Python35\lib\site-packages\urllib3\connection.py", line 150, in 
 _new_conn
 self, "Failed to establish a new connection: %s" % e)
 urllib3.exceptions.NewConnectionError: 
 <urllib3.connection.VerifiedHTTPSConnection object at 0x0000000003694390>: 
 Failed to establish a new connection: [WinError 10060] A connection attempt 
 failed because the connected party did not properly respond after a period 
 of time, or established connection failed because connected host has failed 
 to respond

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
 File "C:\Python35\lib\site-packages\requests\adapters.py", line 440, in 
 send
 timeout=timeout
 File "C:\Python35\lib\site-packages\urllib3\connectionpool.py", line 649, 
 in urlopen
 _stacktrace=sys.exc_info()[2])
 File "C:\Python35\lib\site-packages\urllib3\util\retry.py", line 388, in 
 increment
 raise MaxRetryError(_pool, url, error or ResponseError(cause))
 urllib3.exceptions.MaxRetryError: 
 HTTPSConnectionPool(host='stackoverflow.com', port=443): Max retries 
 exceeded with url: / (Caused by 
 NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 
 0x0000000003694390>: Failed to establish a new connection: [WinError 10060] 
 A 
 connection attempt failed because the connected party did not properly 
 respond 
 after a period of time, or established connection failed because connected 
 host has failed to respond',))

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
  File "C:/Users/mwirzonw/PycharmProjects/untitled/test.py", line 4, in 
  <module>
  proxies={"http": "http://proxy.com:9"})
 File "C:\Python35\lib\site-packages\requests\api.py", line 72, in get
   return request('get', url, params=params, **kwargs)
  File "C:\Python35\lib\site-packages\requests\api.py", line 58, in request
   return session.request(method=method, url=url, **kwargs)
  File "C:\Python35\lib\site-packages\requests\sessions.py", line 502, in 
  request
  resp = self.send(prep, **send_kwargs)
  File "C:\Python35\lib\site-packages\requests\sessions.py", line 612, in 
  send
  r = adapter.send(request, **kwargs)
  File "C:\Python35\lib\site-packages\requests\adapters.py", line 504, in 
  send
   raise ConnectionError(e, request=request)
   requests.exceptions.ConnectionError: 
   HTTPSConnectionPool(host='stackoverflow.com', port=443): Max retries 
   exceeded with url: / (Caused by 
   NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 
   0x0000000003694390>: Failed to establish a new connection: [WinError 
   10060] A connection attempt failed because the connected party did not 
   properly respond after a period of time, or established connection failed 
   because connected host has failed to respond',))

   Process finished with exit code 1

I really need help from who read this post. Because I'm new to python and I've tried a lot of methods.

I've tried using urllib2, urllib3, using request based on this post-Using an HTTP PROXY - Python

But still get the same output.

Thank you....

Fiqri Mfbw
  • 243
  • 2
  • 5
  • 12
  • what is the url that gives the error? where is the code for calling that request? we can't reproduce the error without seeing that – Ofer Sadan Jul 12 '17 at 05:17
  • Is it possible that the proxy is blocking the request? Have you tried it from the browser first? – John Scattergood Jul 12 '17 at 05:26
  • On the top of my question, I already posted my coding. its requests from the stack URL and bypass by using the proxy and read all the content of the URL @OferSadan – Fiqri Mfbw Jul 12 '17 at 05:47
  • Yaa, but I have included the proxy to bypass from blocking. It's weird because when comes to Google URL it successful to show the Output. @JohnScattergood – Fiqri Mfbw Jul 12 '17 at 05:51

0 Answers0