1

I recently made changes to my .htaccess file as instructed on http://www.roimarketing.co.nz/articles/what-is-referrer-spam-how-can-you-block-it/

I had been having problems with bots crawling my site and making it hard to interpret analytics on it. After adding the code shown in the link above, the site takes a minute+ to load each page where-as before it took barely 5 seconds. What did I do wrong, and how can I fix? Chances are I'm making a rookie mistake I've overlooked.

Website in question: royalelectric.com

I edited the .htaccess file from my /public_html/ directory.

Here was my old .htaccess file:

# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress

Here is my new .htaccess file:

## STOP REFERRER SPAM ##
RewriteEngine On
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?4webmasters\.org [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?site1.free-share-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?site2.free-share-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?site3.free-share-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?site4.free-share-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?anticrawler\.org [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?addons\.mozilla\.org [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?baidu\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?bestwebsitesawards\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?best-seo-solution\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?best-seo-offer\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?blackhatworth\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?buttons-for-website\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?buttons-for-your-website\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?buy-cheap-online\.info [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?econom\.co [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?darodar\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?sanjosestartups\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?free-social-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?guardlink\.org [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?event-tracking\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?domination\.ml [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?hulfingtonpost\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?Get-Free-Traffic-Now\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?googlsucks\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?free-share-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?humanorightswatch\.org [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?ilovevitaly\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?7makemoneyonline\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?o-o-6-o-o\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?priceg\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?social-buttons\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?semalt\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?smailik\.org [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?theguardlan\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?simple-share-buttons\.com [NC]
RewriteRule .* – [F]
## STOP REFERRER SPAM ##


# BEGIN WordPress
<IfModule mod_rewrite.c>
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

# END WordPress

Any help you can give me here would be greatly appreciated, as I'm sick of dealing with the bots screwing up my analytics but I don't want to sacrifice load times to fix the problem. Thanks!

bmbu924
  • 25
  • 6
  • It seems extremely unlikely that adding that stack of `RewriteCond` to detect crawlers has slowed your site so drastically, even on a really terrible and resource constrained Apache server. I have to ask - have you commented it out to verify that it is really the .htaccess causing the performance problem? – Michael Berkowski May 28 '15 at 14:54
  • As soon as I made the changes to the .htaccess file the site slowed down. earlier this morning it was working/loading fast as it had been. I've tried putting the new code above and below the existing code and there doesnt seem to be a difference in load times there. – bmbu924 May 28 '15 at 14:56
  • I just loaded your site and watched the network console (it loads in around 10s for me). You have a great many image resources loading, and Apache would be processing the RewriteCond for every image request. If you comment out the new rules and it is fast again it would be possible to tell Apache to skip all the referer checks for .png, .jpg, .gif resources. – Michael Berkowski May 28 '15 at 14:58
  • I'm pretty new to dealing with anything outside of html and css, but what would the snippet be that would skip all the referer checks for the .png, .jpg, .gif resources.? Thanks for your help. – bmbu924 May 28 '15 at 15:07
  • Is This It? RewriteRule .*\.jpg - [S] RewriteRule .*\.png - [S] RewriteRule .*\.jpeg - [S] RewriteRule .*\.gif - [S] – bmbu924 May 28 '15 at 15:16
  • As the first `RewriteCond` in the list after `RewriteEngine`, you might try something like `RewriteCond %{REQUEST_URI} !\.(jpg|png|gif|css) [NC]`. The `!` inverts the match, so the subsequent conditions will only be matched for non-images/css. – Michael Berkowski May 28 '15 at 15:16
  • Ok great thanks so much – bmbu924 May 28 '15 at 15:17

1 Answers1

0

The simple rewrite rules shouldn't slow down your website. Unless there is some misplaced code causing it, check if the rest of the code is ok, and since this file is very sensitive always make a backup before editing it, because it can shut down your site.

Most of those lines are useless because your are trying to block something that doesn't access your site. The only ones that are really useful are these 5 from crawlers

## STOP REFERRER SPAM ##
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?best-seo-solution\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?best-seo-offer\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?7makemoneyonline\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?buttons-for-website\.com [NC,OR]
RewriteCond %{HTTP_REFERER} ^([^.]+.)*?buttons-for-your-website\.com [NC]
RewriteRule .* – [F]
## STOP REFERRER SPAM ##

To keep clean your .htaccess file, you should delete rest because that's ghost spam, it doesn't have any interaction with your website, and that's what these rules does, block the access to your website.

The ghost spam hits your reports directly, so to stop them the only solution is to create a filter in Google Analytics.

You can find more information about the solution and the spam here

https://stackoverflow.com/a/28354319/3197362

https://stackoverflow.com/a/29645325/3197362

Community
  • 1
  • 1
Carlos Escalera Alonso
  • 2,333
  • 2
  • 25
  • 37