2

There are some bots that just keep hitting my site over and over again.

http://proxy.parser.by/proxy.php (Referrer)

I don't want to even reply to anyone that is requesting a .php, or .htm, or .html file.

What is the best way of not responding to such requests?

Update: (I don't want to incur the cost of responding)

Val Neekman
  • 17,692
  • 14
  • 63
  • 66

2 Answers2

3

This is probably best done at the server level, before the request even gets to Django. For example, in Apache, you can use mod_rewrite for access control. This rule rejects all requests with paths ending with .php, .htm, or .html:

RewriteRule \.(php|html?)$ - [F]

The Apache documentation explains how to block requests by user agent, by referer, by orginating IP address, and so on.

Gareth Rees
  • 64,967
  • 9
  • 133
  • 163
  • I don't want to reply at all. – Val Neekman Feb 17 '12 at 20:44
  • Then I think you need a firewall. Not an area I know much about, but maybe take a look at [ModSecurity](http://www.modsecurity.org/)? But the [Apache wiki says](http://wiki.apache.org/httpd/ProxyAbuse) "the savings in resource usage [from dropping requests instead of rejecting them] will be minuscule" – Gareth Rees Feb 17 '12 at 20:49
0

Drop in some middleware which will detect any request you don't want to handle and have it return an HTTP 403 (Forbidden).

David Wolever
  • 148,955
  • 89
  • 346
  • 502