8

How would you split a domain name that will return name and extension

SilentGhost
  • 307,395
  • 66
  • 306
  • 293
bobsr
  • 3,879
  • 8
  • 30
  • 37
  • This stackoverflow question may be useful: http://stackoverflow.com/questions/1066933/python-extract-domain-name-from-url – Gary Kerr May 26 '10 at 09:22

9 Answers9

22

Wow, there are a lot of bad answers here. You can only do this if you know what's on the public suffix list. If you are using split or a regex or something else, you're doing this wrong.

Luckily, this is python, and there's a library for this: https://pypi.python.org/pypi/tldextract

From their readme: 

>>> import tldextract
>>> tldextract.extract('http://forums.news.cnn.com/')
ExtractResult(subdomain='forums.news', domain='cnn', suffix='com')

ExtractResult is a namedtuple. Makes it pretty easy.

The advantage of using a library like this is that they will keep up with the additions to the public suffix list so you don't have to.

mlissner
  • 17,359
  • 18
  • 106
  • 169
  • This would be nice if it worked reliably, but it doesn't work reliably AT ALL. Half the time it thinks the domain is part of the hostname and other times it thinks the suffix is the domain. – Nathan McKaskle May 10 '19 at 20:41
  • 2
    Can you provide some examples? – mlissner May 11 '19 at 00:01
  • We have a domain called clmlb.loc where it decided that machines named: servername.clmlb.loc - the domain was loc while the clmlb was the servername and it dropped the servername entirely. I had an easier time just using the split method in this case. In any case I only wanted xxxx.xxx at the end anyway. – Nathan McKaskle May 13 '19 at 20:28
  • 3
    Well...um, `.loc` isn't a real suffix, is it? This only works on real domain names. – mlissner May 14 '19 at 00:08
  • Thank you! This is a great lib for this job – GregC Jun 11 '20 at 09:43
  • If you know your psuedo-TLD is ".loc" ... can't you just roll your own wrapper to do `hostname.replace('.loc','.com')` on the way in and reverse on the way out... could even use monkey-patching to make it transparent (until tldextract supports psuedo TLDs, unless it already does...) – qneill Jul 19 '20 at 02:23
  • 1
    Great answer. I faced this issue too. Sometimes you get something like yahoo.com.sg. Using regex, yahoo will be considered the subdomain, com the rot and sg the extension which isn't right. – Paschal Sep 06 '21 at 22:45
6

In general, it's not easy to work out where the user-registered bit ends and the registry bit begins. For example: a.com, b.co.uk, c.us, d.ca.us, e.uk.com, f.pvt.k12.wy.us...

The nice people at Mozilla have a project dedicated to listing domain suffixes under which the public can register domains: http://publicsuffix.org/

Andrew Aylett
  • 39,182
  • 5
  • 68
  • 95
4

Depending on your application, be a little wary of simply taking the part following the last '.'. That works fine for .com, .net, .org, etc but will likely fall over for many County Code TLDs. E.g. bit.ly or google.co.uk.

(By which I mean 'bit.ly' probably prefer to be identified including the .ly TLD whereas google probably don't want to be identified with a spurious .co remainder. Whether that's important will obviously depend on what you're doing).

In those complicated cases ... well, you've got your work cut out I suspect!

A robust answer will probably depend on how you're gathering / storing your domains and what you really want back as the 'name'.

For example, if you've got a set of domain names, with no subdomain information, then you could do the opposite of what's suggested above and simply take the first part off:

>>> "stackoverflow.com".split('.')[0]
'stackoverflow'
pycruft
  • 66,157
  • 1
  • 20
  • 12
2

You mean internet domain name, like www.stackoverflow.com? If so, then just use:

>>> 'www.stackoverflow.com'.rsplit('.', 1)
['www.stackoverflow', 'com']
Tomasz Zieliński
  • 16,136
  • 7
  • 59
  • 83
2

As other commenters have pointed out, there's no surefire way of doing this, short of having a dynamically updated list of TLDs and gTLDs. What works for google.com might not work for google.co.uk, or something.co.xx, or something.com.xx. Pretty much anything can be in a TLD or a gTLD and who knows what the future holds?

So there are two very different approaches:

  1. Use a library that has a regularly-updated list of TLDs and gTLDs, like tldextract.
  2. Use an algorithm that you know will fail on some edge-cases, but aim for as few as possible.

In my experience, the following satisfies #2 well, assuming you've already stripped off protocol and path:

def domain_from_hostname( hostname ):
    # Assume any TLD business will be dealt with in the last 7 bytes
    # This won't work for:
    # - Subdomains on supershort domains with short TLDs
    # - Domains with TLDs over 7 bytes long (".diamonds" anyone?) 
    if len(host) < 8:
        return host.strip( 'www.' )

    host_left = host[:-7].split('.')[-1]
    return u'%s%s' % ( host_left, host[-7:] )

Try it with some weirdness: a .com.au, .media, .in, .中信 etc.

Aidan Fitzpatrick
  • 1,950
  • 1
  • 21
  • 26
0
domain = 'subdomain.domain.ext'
name, ext = domain.split('.')[-2:]
Johannes Charra
  • 29,455
  • 6
  • 42
  • 51
  • 1
    This only works for top-level US domains, in the form .com, .org, .ext etc. For other regions use the tldextract library as described in mlissner's answer below. – Jon C Nov 26 '19 at 10:07
  • domain = 'http://google.com/' name, ext = domain.split('.')[-2:] print(name) print(ext) # Here it print as name: "http://google" domain = "com/ – prasadbhagwat Sep 14 '21 at 03:41
0

If you always want to get the last part of a domain name, you can:

subdomain, _, domain= fqdn.rpartition('.')
tzot
  • 92,761
  • 29
  • 141
  • 204
0

I guess you will find urlparse module interesting: http://docs.python.org/library/urlparse.html

dmedvinsky
  • 8,046
  • 3
  • 31
  • 25
0

This is what I have come up with. Nothing fancy. It works for me. Although I do believe it gives weird feedback sometimes when there is characters like ?, +, so on. Still don't understand why.

scheme = 'https://www.msn.com/d/1234.php?=https://www.msn.com?+'
notfound = -1
https = scheme.rfind('http://')
com = scheme.rfind('.com')
if https != notfound:
    if com != notfound:
        domain = scheme[https:com+len('.com')]
        return scheme[https:com+len('.com')]

#Here we can grab the double suffix. This one has not been fully tested.

def getdoublesuffix(domain):
    '''
    :description: returns double dot TLD suffix endings or returns -1
    :function: 
    '''
    # ['www.domain.co.com'] to
    # ['www.domain', 'co', 'com']
    dots = domain.rsplit(sep='.', maxsplit=2)
# count dots by enumeration not string count! Only interested in enumeration count and
# not total dot count since it is split by '.' as a separator.
    for number, value in enumerate(dots, 0):
        value = value
        number = number
    if number is 2:
        # co.com
        result = '{0}.{1}'.format(dots[1], dots[2])
        return result
    else:
        #return that we do not have a domain ending in two dot notation.
        return -1
Nightforce2
  • 1,426
  • 5
  • 18
  • 31