How to block referral spam using Nginx?

Question

I'm running two mongrels under an Nginx server. I keep getting requests for a nonexistent file. The IP addresses change frequently but the referring URL stays the same. I'd like to resolve this.

Answer 1

Using Nginx map module is a a bit more efficient and easier to manage as the list gets long.

Put this in your http {} block :

map $http_referer $bad_referer {
    hostnames;

    default                           0;

    # Put regexes for undesired referers here
    "~social-buttons.com"             1;
    "~semalt.com"                     1;
    "~kambasoft.com"                  1;
    "~savetubevideo.com"              1;
    "~descargar-musica-gratis.net"    1;
    "~7makemoneyonline.com"           1;
    "~baixar-musicas-gratis.com"      1;
    "~iloveitaly.com"                 1;
    "~ilovevitaly.ru"                 1;
    "~fbdownloader.com"               1;
    "~econom.co"                      1;
    "~buttons-for-website.com"        1;
    "~buttons-for-your-website.com"   1;
    "~srecorder.co"                   1;
    "~darodar.com"                    1;
    "~priceg.com"                     1;
    "~blackhatworth.com"              1;
    "~adviceforum.info"               1;
    "~hulfingtonpost.com"             1;
    "~best-seo-solution.com"          1;
    "~googlsucks.com"                 1;
    "~theguardlan.com"                1;
    "~i-x.wiki"                       1;
    "~buy-cheap-online.info"          1;
    "~Get-Free-Traffic-Now.com"       1;
}

Put this in your server {} block:

if ($bad_referer) { 
    return 444; # emtpy response
}

It worked for me.

Got this from http://fadeit.dk/blog/post/nginx-referer-spam-blacklist

Answer 2

https://calomel.org/nginx.html

Block most "referrer spam" -- "more of an annoyance than a problem"

nginx.conf

    ## Deny certain Referers (case insensitive)
    ## The ~* makes it case insensitive as opposed to just a ~
 if ($http_referer ~* (babes|click|diamond|forsale|girl|jewelry|love|nudit|organic|poker|porn|poweroversoftware|sex|teen|video|webcam|zippo))
    {  return 403;   }

Answer 3

I have created module for checking incoming IP in black lists https://github.com/oneumyvakin/ngx_http_blacklist_lookup_module

it's uses blacklists from projecthoneypot.org, blocklist.de and uceprotect.net

Answer 4

I've been in a similar situation before where I needed to block people based on behaviour instead of other arbitrary rules that a firewall could sort out on its own.

They way I worked around the problem was to make my logic (Rails in your case) do the blocking... But a long way round:

• Have your logic maintain a block-list as a new-line separated plaintext file.
• Create a bash (or other) script as root to read this file and add its listees to your firewall's blocklist
• Create a cron job to call the script, again, as root

The reason I do it this way around (rather than just giving Django permissions to alter firewall config) is simply: security. If my application were hacked, I wouldn't want it to hurt anything else.

The bash script is something like this:

exec < /path/to/my/djago-maintained/block-list
while read line
do

    iptables -A INPUT --source $line/32 -j DROP

done