9

Does anyone know a regular expression to match Domain.CCTLD? I don't want subdomains, only the "atomic domain". For example, docs.google.com doesn't get matched, but google.com does. However, this gets complicated with stuff like .co.uk, CCTLDs. Does anyone know a solution? Thanks in advance.

EDIT: I've realized I also have to deal with multiple subdomains, like john.doe.google.co.uk. Need a solution now more than ever :P.

Tom
  • 283
  • 1
  • 4
  • 14
  • Do you explicitly need a regex, or would a function to do it suffice? – Benson Jul 07 '10 at 22:18
  • This would become a quite large regex, seeing as you would need to treat all ccSLDs as special cases, and there are a lot (and I mean A LOT) of ccSLDs. Brazil has 66 of them! – You Jul 07 '10 at 22:23
  • @Benson, a function would work, as long as it could find domain.cctld in a long list of domains – Tom Jul 07 '10 at 22:26
  • possible duplicate of [Get the subdomain from a URL](http://stackoverflow.com/questions/288810/get-the-subdomain-from-a-url) – Greg Hewgill Jul 08 '10 at 03:01

3 Answers3

8

It sounds like you are looking for the information available through the Public Suffix List project.

A "public suffix" is one under which Internet users can directly register names. Some examples of public suffixes are ".com", ".co.uk" and "pvt.k12.wy.us". The Public Suffix List is a list of all known public suffixes.

There is no single regular expression that will reasonably match the list of public suffixes. You will need to implement code to use the public suffix list, or find an existing library that already does so.

Greg Hewgill
  • 951,095
  • 183
  • 1,149
  • 1,285
  • Interesting and probably very useful list. – You Jul 07 '10 at 22:29
  • 1
    Thanks, Greg. That's absolutely the right answer. There are libraries to do Public Suffix List processing in several languages at http://www.dkim-reputation.org/regdom-libs/ – Anirvan Jul 07 '10 at 23:17
  • @Anirvan, do you know an equivalent for Python? The library you posted is only available in C, PHP, and Perl. – Tom Jul 07 '10 at 23:26
  • @Tom: Over a year later, here is a python package for the job: http://pypi.python.org/pypi/publicsuffix – Martijn Pieters Oct 14 '11 at 10:54
3

Based on your comment above, I'm going to reinterpret the question -- rather than making a regex that will match them, we'll create a function that will match them, and apply that function to filter a list of domain names to only include first class domains, e.g. google.com, amazon.co.uk.

First, we'll need a list of TLDs. As Greg mentioned, the public suffix list is a great place to start. Let's assume you've parsed the list into a python array called suffixes. If this isn't something your comfortable with, comment and I can add some code that will do it. 

suffixes = parse_suffix_list("suffix_list.txt")

Now we'll need code that identifies whether a given domain name matches the pattern some-name.suffix:

def is_domain(d):
    for suffix in suffixes:
        if d.endswith(suffix):
            # Get the base domain name without suffix
            base_name = d[0:-(suffix.length + 1)]
            # If it contains '.', it's a subdomain. 
            if not base_name.contains('.'):
                return true
    # If we get here, no matches were found
    return false
Benson
  • 22,457
  • 2
  • 40
  • 49
2

I would probably solve this by getting a complete list of TLDs and using it to create the regex. For example (in Ruby, sorry, not a Pythonista yet):

tld_alternation = ['\.com','\.co\.uk','\.eu','\.org',...].join('|')
regex = /^[a-z0-9]([a-z0-9\-]*[a-z0-9])?(#{tld_alternation})$/i

I don't think it's possible to properly differentiate between a real two part TLD and a subdomain without knowing the actual list of TLDs (ie: you could always construct a subdomain that looks like a TLD if you knew how the regex worked.)

Mike Tunnicliffe
  • 10,674
  • 3
  • 31
  • 46