2

Hello everyone I am pretty new to programming so please forgive me any noob mistakes. I am trying to use urllib in python 3.6 to find a string on a webpage and the solution I found here is working great for most webpages i tested, for example:

from urllib.request import urlopen
>>>string = 'google'
>>>url = 'https://google.com/'
>>>webp=urlopen(url).read()
>>>isfound = webp.find(string.encode('utf-8'))
>>>print(isfound)
186

But when I try using the URL https://chan.sankakucomplex.com/ (NSFW) I get tons of errors and I don't know why. I allready tried changing my DNS to 8.8.8.8 but this didn't change anything.

error: Traceback (most recent call last): File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 1318, in do_open encode_chunked=req.has_header('Transfer-encoding')) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1239, in request self._send_request(method, url, body, headers, encode_chunked) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1285, in _send_request self.endheaders(body, encode_chunked=encode_chunked) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1234, in endheaders self._send_output(message_body, encode_chunked=encode_chunked) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1026, in _send_output self.send(msg) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 964, in send self.connect() File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 1392, in connect super().connect() File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 936, in connect (self.host,self.port), self.timeout, self.source_address) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\socket.py", line 704, in create_connection for res in getaddrinfo(host, port, 0, SOCK_STREAM): File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\socket.py", line 743, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno 11004] getaddrinfo failed During handling of the above exception, another exception occurred: Traceback (most recent call last): File "C:/Users/-/Desktop/python/searcher/stackex.py", line 5, in <module> webp=urlopen(url).read() File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 223, in urlopen return opener.open(url, data, timeout) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 526, in open response = self._open(req, data) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 544, in _open '_open', req) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 504, in _call_chain result = func(*args) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 1361, in https_open context=self._context, check_hostname=self._check_hostname) File "C:\Users\-\AppData\Local\Programs\Python\Python36\lib\urllib\request.py", line 1320, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [Errno 11004] getaddrinfo failed>

moequasar
  • 23
  • 3
  • are you perhaps in a country or in some network that blocks this content? – Azsgy Apr 05 '18 at 11:58
  • Your code works on my machine (with your website and the string "sankaku". [Similar issue](https://stackoverflow.com/questions/35835767/urllib-error-urlerror-urlopen-error-errno-11002-getaddrinfo-failed). – Ettore Rizza Apr 05 '18 at 11:59
  • Thank you! I forgot I needed to use my proxy to access the site. Any idea how I get urllib to use my SOCKS5 proxy? This http://dae.me/blog/1959/using-pythons-urllib2-or-requests-with-a-socks5-proxy/ doesn't seem to work for me. – moequasar Apr 05 '18 at 12:44

1 Answers1

2
urllib.error.URLError: <urlopen error [Errno 11004] getaddrinfo failed>

This tells me that your DNS server is refusing to translate the text address (chan.sankakucomplex.com) into an IP address. It's not an issue with Python.

Can you get to that site from a web browser running on the same machine where you're running your python code?

Chris Curvey
  • 9,738
  • 10
  • 48
  • 70
  • Thank you! I forgot I needed to use my proxy to access the site. Any idea how I get urllib to use my SOCKS5 proxy? This http://dae.me/blog/1959/using-pythons-urllib2-or-requests-with-a-socks5-proxy/ doesn't seem to work for me. – moequasar Apr 05 '18 at 12:47
  • 1
    not sure about urllib. But you'll have a much easier life if you start using requests (http://docs.python-requests.org/en/master/) – Chris Curvey Apr 05 '18 at 13:56