4

I was trying to send http/https requests via proxy (socks5), but I can't understand if the problem is in my code or in the proxy.

I tried using this code and it gives me an error:

requests.exceptions.ConnectionError: SOCKSHTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x000001B656AC9608>: Failed to establish a new connection: Connection closed unexpectedly'))

This is my code:

import requests

url = "https://www.google.com"


proxies = {
    "http":"socks5://fsagsa:sacesf241_country-darwedafs_session-421dsafsa@x.xxx.xxx.xx:31112",
    "https":"socks5://fsagsa:sacesf241_country-darwedafs_session-421dsafsa@x.xxx.xxx.xx:31112",
    }

headers = {
        "Upgrade-Insecure-Requests": "1",
        "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4577.63 Safari/537.36",
        "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "Sec-Gpc": "1",
        "Sec-Fetch-Site": "same-origin",
        "Sec-Fetch-User": "?1",
        "Accept-Encoding": "gzip, deflate, br",
        "Sec-Fetch-Mode": "navigate",
        "Sec-Fetch-Dest": "document",
        "Accept-Language": "en-GB,en;q=0.9"
    }


r = requests.get(url, headers = headers, proxies = proxies)

print(r)

Then, I checked the proxy with an online tool The tool manages to send requests through the proxy. Online tool output.

So the problem is in this code? I can't figure out what's wrong.

Edit (15/09/2021)

I added headers but the problem is still there.

SAL
  • 547
  • 2
  • 8
  • 25
  • 1
    have you tried adding your headers to tell your site what you want? : `accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9` `accept-encoding: gzip, deflate, br` `accept-language: en-GB,en;q=0.9` – RichEdwards Sep 13 '21 at 09:12
  • Yes, I have tried. Unfortunately it didn't solve my problem. – SAL Sep 13 '21 at 09:21

2 Answers2

6

Create a local server/mock to handle the request using pytest or some other testing framework with responses library to eliminate variables external to your application/script. I’m quite sure Google will reject requests with empty headers. Also, ensure you installed the correct dependencies to enable SOCKS proxy support in requests (python -m pip install requests[socks]). Furthermore, if you are making a remote request to connect to your proxy you must change socks5 to socks5h in your proxies dictionary.

References

pytest: https://docs.pytest.org/en/6.2.x/

responses: https://github.com/getsentry/responses

requests[socks]: https://docs.python-requests.org/en/master/user/advanced/#socks

pygeek
  • 7,356
  • 1
  • 20
  • 41
  • I've already tried to add headers to that request and also, I've already tried all the steps you mentioned. – SAL Sep 14 '21 at 16:51
4

In addition to basic HTTP proxies, Requests also supports proxies using the SOCKS protocol. This is an optional feature that requires that additional third-party libraries be installed before use.

You can get the dependencies for this feature from pip:

$ python -m pip install requests[socks]

Once you’ve installed those dependencies, using a SOCKS proxy is just as easy as using a HTTP one:

proxies = {
            'http': 'socks5://user:pass@host:port',
            'https': 'socks5://user:pass@host:port'
          }

Using the scheme socks5 causes the DNS resolution to happen on the client, rather than on the proxy server. This is in line with curl, which uses the scheme to decide whether to do the DNS resolution on the client or proxy. If you want to resolve the domains on the proxy server, use socks5h as the scheme.

TomCat
  • 83
  • 7