11

I'm trying to use the new socket support for Google App Engine in order to perform some DNS queries. I'm using dnspython to perform the query, and the code works fine outside GAE.

The code is the following:

class DnsQuery(webapp2.RequestHandler):

    def get(self):
       domain  = self.request.get('domain')
       logging.info("Test Query for "+domain)
       answers = dns.resolver.query(domain, 'TXT', tcp=True)
       logging.info("DNS OK")
       for rdata in answers:
          rc =  str(rdata.exchange).lower()
          logging.info("Record "+rc)

When I run in GAE I get the following error:

  File "/base/data/home/apps/s~/one.366576281491296772/main.py", line 37, in post
    return self.get()   
  File "/base/data/home/apps/s~/one.366576281491296772/main.py", line 41, in get
    answers = dns.resolver.query(domain, 'TXT', tcp=True)
  File "/base/data/home/apps/s~/one.366576281491296772/dns/resolver.py", line 976, in query
    raise_on_no_answer, source_port)
  File "/base/data/home/apps/s~/one.366576281491296772/dns/resolver.py", line 821, in query
    timeout = self._compute_timeout(start)
  File "/base/data/home/apps/s~/one.366576281491296772/dns/resolver.py", line 735, in _compute_timeout
    raise Timeout

Which is raised by dnspython when no answer is returned within the time limit. I've raised the timelimit to 60 seconds, and DnsQuery is a task, but still getting the same error.

Is there any limitation in Google App Engine socket implementation, which prevents the execution of DNS requests ?

Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97
ROOTto
  • 133
  • 5
  • I also have a similar problem with Google App Engine. – Uri Mar 14 '16 at 14:51
  • @Cyril N: check that in addition to having a paid app you also set a daily budget for it - at least last year there were reports of some paid-only features not working properly until the daily budget was initialized. – Dan Cornilescu Apr 20 '16 at 15:23
  • @DanCornilescu I went to Billing > Budgets & Alerts and I've already defined an alert of certain threesholds. Is that what you meant? I read at http://stackoverflow.com/a/10894376/330867 that `Google will wait until the first payment is cleared before increasing your mail quota.`. Maybe that's my issue. But then I went to my App Engine > Quotas, and I can see that under the Sockets entry, my app did some requests (maybe using the Python package `requests`, which I use). So `Sockets` should work, but not with `dnspython` ? – Cyril N. Apr 20 '16 at 16:42
  • @CyrilN. I mean this: http://stackoverflow.com/questions/27616776/how-do-i-set-a-cost-limit-in-google-developers-console (you need to adjust for the new developer console location, shortcut is https://console.cloud.google.com/appengine/settings?project) – Dan Cornilescu Apr 20 '16 at 17:01
  • @DanCornilescu I have two similar projects (dev/prod) and prod had billing enabled (from your link), so I tested with it but it still does not work :/ – Cyril N. Apr 21 '16 at 10:14
  • Other things to try: check if your app's (or dnspython's) use of sockets conflicts with GAE sockets limitations (ports, IP addresses, etc in https://cloud.google.com/appengine/docs/python/sockets/); or if it's a problem on your DNS server side (does it work with another server or is the same dnspython snippet working from a python app not running in GAE?); or if some other sockets-based code works in your app (to check if your app's paid status impact or not sockets operation) - trying to reduce the complexity of the investigation :) – Dan Cornilescu Apr 21 '16 at 11:01
  • @DanCornilescu I took some swing at the dnspython code to try to see where it would fail but my lack of knowledge of the library didn't helped me. From what I saw, it uses TCP/UDP (allowed), does not connect to FTP, uses port 53 (DNS). The query methods (https://github.com/rthalley/dnspython/blob/99fd864361a4e8f3988440b315d645e16bbf94b7/dns/resolver.py#L777) seems to connect to 0.0.0.0 which may be the issue. I'm going to look into that. – Cyril N. Apr 25 '16 at 08:07
  • No I read that wrong, it's the source that defaults to the local ip address. – Cyril N. Apr 25 '16 at 08:29
  • Depending on what you need it for, and if you need a quick solution, you could use a third-party API instead. I created a spreadsheet for managing some DNS stuff and I am using `dns-api.org` with a simple HTTP request for it. (This is not intended to be a full solution for your problem, but rather a possible workaround.) – CherryDT Apr 26 '16 at 10:59
  • @DanCornilescu I'd like to give you the bounty for your help but I can't give it to a comment, only an answer. If you want to post some help that will increase the quality of the question, I will be more than happy to give them to you. One drawback, there is only a few hours remaining for the bounty :/ – Cyril N. Apr 27 '16 at 19:54
  • @CherryDT Thank you for your suggestion. I also thought about the third party but this solution only add one more layer of communication and is more subject to failures (api down for example). I finally did a workaround and used `gethostbyname` to check if the domain is correct. – Cyril N. Apr 27 '16 at 19:55

3 Answers3

7

This is a bug and will be fixed ASAP.

As a workaround, pass in the source='' argument to dns.resolver.query.

tcp=True is not necessary.

Iain Wade
  • 176
  • 1
1

No. There is no limit on UDP ports. (only smtp ports on TCP).

It is possible there is an issue with the socket service routing. Please file an issue with the app engine issue tracker. https://code.google.com/p/googleappengine/issues/list

ozzee
  • 141
  • 5
1

dnspython is using socket. However, socket is only available in paid apps.1