I've tried a lot way to read the content or extract background HTML from other website using Python 3.5.
I'm familiar with Using an HTTP PROXY, I develop my project on Windows Server 2008 r2, and it's the policy of the company.
Below is my simple code to be able to request connection to the other website.
import requests
r = requests.get("https://stackoverflow.com",
proxies={"http": "http://proxy.com:900"})
print(r.text)
Its work when I change the url to google urls as below
r = requests.get("https://www.google.com",
proxies={"http": "http://proxy.com:900"})
When i use url from another website.
Then, I got this error on the output.
C:\Python35\python.exe C:/Users/mwirzonw/PycharmProjects/untitled/test.py
Traceback (most recent call last):
File "C:\Python35\lib\site-packages\urllib3\connection.py", line 141, in
_new_conn
(self.host, self.port), self.timeout, **extra_kw)
File "C:\Python35\lib\site-packages\urllib3\util\connection.py", line 83,
in create_connection
raise err
File "C:\Python35\lib\site-packages\urllib3\util\connection.py", line 73,
in create_connection
sock.connect(sa)
TimeoutError: [WinError 10060] A connection attempt failed because the
connected party did not properly respond after a period of time, or
established connection failed because connected host has failed to respond
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python35\lib\site-packages\urllib3\connectionpool.py", line 600,
in urlopen
chunked=chunked)
File "C:\Python35\lib\site-packages\urllib3\connectionpool.py", line 345,
in _make_request
self._validate_conn(conn)
File "C:\Python35\lib\site-packages\urllib3\connectionpool.py", line 844,
in
_validate_conn
conn.connect()
File "C:\Python35\lib\site-packages\urllib3\connection.py", line 284, in
connect
conn = self._new_conn()
File "C:\Python35\lib\site-packages\urllib3\connection.py", line 150, in
_new_conn
self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError:
<urllib3.connection.VerifiedHTTPSConnection object at 0x0000000003694390>:
Failed to establish a new connection: [WinError 10060] A connection attempt
failed because the connected party did not properly respond after a period
of time, or established connection failed because connected host has failed
to respond
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python35\lib\site-packages\requests\adapters.py", line 440, in
send
timeout=timeout
File "C:\Python35\lib\site-packages\urllib3\connectionpool.py", line 649,
in urlopen
_stacktrace=sys.exc_info()[2])
File "C:\Python35\lib\site-packages\urllib3\util\retry.py", line 388, in
increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError:
HTTPSConnectionPool(host='stackoverflow.com', port=443): Max retries
exceeded with url: / (Caused by
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at
0x0000000003694390>: Failed to establish a new connection: [WinError 10060]
A
connection attempt failed because the connected party did not properly
respond
after a period of time, or established connection failed because connected
host has failed to respond',))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/mwirzonw/PycharmProjects/untitled/test.py", line 4, in
<module>
proxies={"http": "http://proxy.com:9"})
File "C:\Python35\lib\site-packages\requests\api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "C:\Python35\lib\site-packages\requests\api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python35\lib\site-packages\requests\sessions.py", line 502, in
request
resp = self.send(prep, **send_kwargs)
File "C:\Python35\lib\site-packages\requests\sessions.py", line 612, in
send
r = adapter.send(request, **kwargs)
File "C:\Python35\lib\site-packages\requests\adapters.py", line 504, in
send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError:
HTTPSConnectionPool(host='stackoverflow.com', port=443): Max retries
exceeded with url: / (Caused by
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at
0x0000000003694390>: Failed to establish a new connection: [WinError
10060] A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection failed
because connected host has failed to respond',))
Process finished with exit code 1
I really need help from who read this post. Because I'm new to python and I've tried a lot of methods.
I've tried using urllib2, urllib3, using request based on this post-Using an HTTP PROXY - Python
But still get the same output.
Thank you....