Reverse IP domain check script

Question

I wonder if there is a way to build a automated script (doesnt matter if its perl, bash or php) to find out what are the domains hosted on a certain ip.

I'm trying to do something like the Reverse IP Domain Check on yougetsignal.com.

A friend of mine helped me with this, but it doesn't find the hosted domains given a list of server addresses; instead it just gets the host of the IP address.

Here is the PHP code I have written

<?php

$file    = fopen("71", "r");
$ip_file = fopen('iphosts', 'w');
$ip_add  = fopen('ip_add', 'w');

$i = 1;

while ( ! feof($file) ) {

    $host = fgets($file);
    $host = rtrim($host);

    echo "\n";
    echo "Connecting..." . $host;
    echo "\n";

    $host_name = gethostbyaddr( $host );
    if ( !empty( $host_name ) && $host != $host_name ) {
        fwrite($ip_file, $host_name."\n");
    } else {
        fwrite($ip_add, $host_name."\n");
    }

    echo " Done \n";
    $i++;
}

fclose($file);
fclose($ip_file);
fclose($ip_add);


exit(0);

Answer 1

This is not generally possible. Web sites that host virtual domains do not usually provide reverse DNS that lists all the domains that they host. The reverse DNS just points to their server name.

The site you link to probably gets its information by crawling the web like a search engine. But instead of making an index of all the words in the pages, it simply records all the name-to-address mappings that it finds, and then creates a reverse index that returns all the names that map to the same address.

Answer 2

This is not possible to do from a script. Let me give an example to illustrate.

You have a server located at 12.34.56.78, that server has the following (unique) websites hosted.

www.example.com
example.com
www.example.org
bob.example.net 
gertrudeisarealllyniceguy.banana 

The way that most webservers operate is that they will serve the website that matches the incoming Host header. So you may, luckily discover the existence of www.example.com, you may infer example.com. bob.example.net is more likely to be missed.

So unless you specifically request gertrudeisarealllyniceguy.banana you will never know it exists. So the only possible way you could know every website hosted on that webserver (aside from hacking it) would be to try every single possible domain name, and that would take a very long time (perhaps eons)

Sites that currently have this information use data from web crawlers, they simply look up the columns of the table that has domain name and IP address.

Answer 3

I'm the creator of host.io, which does something similar, showing you a list of all of the domains hosted on the same IP address (along with a list of domains that link to the domain, and more). For example, here's a list of domains hosted on the same IP as stackoverflow.com: https://host.io/stackoverflow.com

As you've discovered, getting the IP address for a single domain is only a very small part of the solution. There is no single command or script you can write to do this - you need to build out your own database of domains to IP address.

First need to get (or create) a list of all available domain names. There are roughly 250 million currently. The next step is to resolve all of those domains to an IP address. You then need to store all of those domain to IP pairs in a database, and then you can query to get a list of all domains on the same IP. And then you need to do that at a regular frequency to make sure it stays up to date.

To give a full example, let's create a file with 4 domains and resolve them to IP addresses:

$ cat domains.txt
facebook.com
fb.com
stackoverflow.com
stackexchange.com

# Let's resolve the domains to IPs with dig - could use nslookup or similar
$ cat domains.txt | xargs -I% bash -c "dig +short % | tail -n1" > ips.txt
31.13.76.68
31.13.76.68
151.101.129.69
151.101.193.69

# Let's combine the domains and IPs using paste
$ paste domains.txt ips.txt > combined.tsv
$ cat combined.tsv
facebook.com    31.13.76.68
fb.com  31.13.76.68
stackoverflow.com   151.101.129.69
stackexchange.com   151.101.129.69

# Let's create a DB table and import the data, and write a query 
# to find any domains in our dataset that are hosted on the same 
# domain as stackoverflow.com

$ psql $DB_URL

=> create table details (domain text, ip text);
=> \copy details from ~/combined.tsv;

=> select domain from details where ip = (select ip from details where domain = 'stackoverflow.com');
      domain
-------------------
 stackoverflow.com
 stackexchange.com
(2 rows)

That's how you could build your own, or you could let someone else do the hard work, and use their data. We're one such provider, but others exist, like yougetsignal and domaintools.