Non-blocking hostname resolving?

Question

Looking for a way how to resolve IPv4/IPv6 address for a given fully qualified domain name, however in a limited time.

I've tried several ways with a standard Ruby library, but all of them block other threads so timeout exception can't take effect:

Timeout.timeout(1) {addr = Socket.sockaddr_in 80,'google.com'}
Timeout.timeout(1) {addr = Socket.getaddrinfo 'google.com','http',nil,:STREAM}
Timeout.timeout(1) {addr = Socket.gethostbyname 'google.com'}

When DNS response is slow or momentarily inaccessible, all of above attempts won't stop sooner then in about 60 seconds instead of given 1 second.

I'm aware of adns but it is not usualy available and requires an extra package to be installed.

Is there some other solution ?

Tymoteusz Paul · Accepted Answer · 2014-10-05T21:21:41.767

2

Easiest way will be to just put the domain query in a thread/process and then periodically check whether we got info back from it.

Edit: As how to do it, here is a simple example in python:

from multiprocessing import Pool
import socket
import time

def lookup(host):
    return socket.gethostbyname(host)

if __name__ == "__main__":
    pool = Pool(processes=4)
    result = pool.apply_async(lookup, ("google.com",))
    done = 0
    while done < 60: # Here we do something weird while domain lookups work in background
        print(done)
        done += 1
        time.sleep(1)

    # Now we need the domain details so we get em
    print(result.get(1)) # 1 second timeout since it already had 60 seconds to process

Gil has nothing to do with it as the lookup function will be executed in separate process, with its own execution stack. You can read up more about your options in regards of multiprocessing on excellent python manpages.

edited Oct 05 '14 at 21:21

answered Oct 05 '14 at 19:06

Tymoteusz Paul

2,732
17
20

I'm afraid it unfortunately won't work as above functions block other threads until finished, so it's not possible to periodically check its state. Tried with CRuby and CPython's `socket.gethostbyname`. It's due GIL that prevents multiple threads during external C library call. – David Unric Oct 05 '14 at 21:04
@DavidUnric i've updated the answer with some details and a multiprocessing example how to achieve what you are trying to do. – Tymoteusz Paul Oct 05 '14 at 21:22
Thanks. Your example does work but instead of a new thread a new separate process is forked. Is this required or would it work in a thread also ? I'm sorry for asking, my primary language is Ruby and not sure how to properly test in Python. – David Unric Oct 05 '14 at 21:35
@DavidUnric no worries, happy to help. And yes, threading will work as it comes with it's own execution stack. You can read up online about the two and how do they differentiate, for example here: http://stackoverflow.com/questions/3044580/multiprocessing-vs-threading-python But in this case any of them will do, and they are a concept bound to the OS rather than language (usually!). – Tymoteusz Paul Oct 05 '14 at 21:38
Great. I did rewrite your example with `multiprocessing.pool.ThreadPool` and it still does work. I'd yet need to rewrite it in Ruby, hope it would work similar. But I'm marking your answer as accepted, as solution either in python or ruby was asked. – David Unric Oct 05 '14 at 21:58
@DavidUnric i did dabble in ruby a bit and used the parallel package with good results so you may want to look in that direction. – Tymoteusz Paul Oct 05 '14 at 22:00

Non-blocking hostname resolving?

1 Answers1