1

I know this has been asked a lot on here but I still can't find an answer that works for my website. These are the posts/answers I've tried:

I am getting duplicate content errors showing up, here is an example

https://www.example.co.uk/coffee-machines/
http://example.co.uk/coffee-machines/
http://www.example.co.uk/coffee-machines/
http://example.co.uk/coffee-machines/

These URLs are showing up as duplicate content so I am trying to force a 301 redirect to

https://www.example.co.uk/coffee-machines/

and not just on that url but site wide. I have a "Canonical" tag set up that points to the correct version but I'd also like to 301 redirect all web pages/directories to the www/https version of the page.

This is my current contents of .htaccess

Options -Indexes

Options +FollowSymLinks

Options -MultiViews

AddDefaultCharset UTF-8

ErrorDocument 401 /401.php
ErrorDocument 404 /404.php

RewriteEngine on
RewriteBase /

RewriteCond %{HTTPS} off
RewriteCond %{HTTP_HOST} !^www\.(.*)$ [NC]
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]

RewriteCond %{HTTPS} on
RewriteCond %{HTTP_HOST} !^www\.(.*)$ [NC]
RewriteRule ^(.*)$ https://www.%{HTTP_HOST}/$1 [R=301,L]

and this works for redirecting example.co.uk to the www/https version but no sub-pages.

Could someone please point me in the right direction to either redirect all urls to the www/https version of suggest a solution to stop the duplicate content.

I've changed the .htaccess to the code below to reflect the post being flagged as duplicate. So the .htaccess now reflects the answer in this post (Redirect non-www and non-http to https) and also included the entire .htaccess file contents.

I am checking the headers by starting a session with putty and then using (curl --head http://example.co.uk) as suggested by @StephenOstermiller

Options -Indexes

Options +FollowSymLinks

Options -MultiViews

AddDefaultCharset UTF-8

ErrorDocument 401 /401.php
ErrorDocument 404 /404.php

RewriteEngine on
RewriteBase /

RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} !^www\.
RewriteRule ^ https://www.xxx.co.uk%{REQUEST_URI} [NC,L,R=301,NE]

RewriteRule ^q&a/$ forum.php

RewriteRule ^q&a/(.*).html$ forumDetail.php?question=$1&%{QUERY_STRING} [B,L]

#RewriteRule ^q&a/(.*)$ /forum/$1 [R=301,NC,L]

RewriteRule ^blog/$ blog.php

RewriteRule ^blog/(.*).html$ blogDetail.php?blog_name=$1&%{QUERY_STRING} [B,L]

RewriteRule ^about-us/$ about.php
RewriteRule ^privacy-policy/$ privacy.php
RewriteRule ^disclaimer/$ disclaimer.php
RewriteRule ^my-account/$ user_account.php
RewriteRule ^activate-my-account/$ user_activate.php
RewriteRule ^update-my-account/$ user_change.php
RewriteRule ^login/$ user_login.php
RewriteRule ^logout/$ user_logout.php
RewriteRule ^register/$ user_register.php
RewriteRule ^reset-my-password/$ user_reset.php
RewriteRule ^home-energy/$ home-energy.php

RewriteRule ^compare/(.*)$ /kitchen-scales/$1 [R=301,NC,L]

RewriteRule ^kitchen_scales/(.*)$ /kitchen-scales/$1 [R=301,NC,L]
RewriteRule ^coffee_machines/(.*)$ /coffee-machines/$1 [R=301,NC,L]
RewriteRule ^digital_radios/(.*)$ /digital-radios/$1 [R=301,NC,L]
RewriteRule ^mixers_blenders/(.*)$ /mixers-blenders/$1 [R=301,NC,L]

RewriteRule ^4g-broadband-all-you-need-to-know.html$ 4g-broadband-all-you-need-to-know.php

Redirect 301 /kitchen-scales/category/digital-scales/index.php https://www.xxx.co.uk/kitchen-scales/category/digital-scales/
Redirect 301 /kitchen-scales/salter-kitchen-scales.html https://www.xxx.co.uk/kitchen-scales/brand/salter/
Redirect 301 /kitchen-scales/brand/index.php https://www.xxx.co.uk/kitchen-scales/brand/
Redirect 301 /kitchen-scales/brand/salter/index.php https://www.xxx.co.uk/kitchen-scales/brand/salter/

Redirect 301 /coffee-machines/category/bean-to-cup-coffee-machines/index.php https://www.xxx.co.uk/coffee-machines/category/bean-to-cup-coffee-machines/
Redirect 301 /coffee-machines/bean-to-cup-coffee-machines.html https://www.xxx.co.uk/coffee-machines/category/bean-to-cup-coffee-machines/
Redirect 301 /coffee-machines/bean-to-cup-coffee-machines.php https://www.xxx.co.uk/coffee-machines/category/bean-to-cup-coffee-machines/

<IfModule mod_headers.c>
  Header set Connection keep-alive
</IfModule>

<FilesMatch "\.(ico|pdf|jpg|jpeg|png|webp|gif|html|htm|xml|txt|xsl|css)$">
Header set Cache-Control "max-age=31536000"
</FilesMatch>

AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript text/javascript

<FilesMatch "config\.php|config\.advanced\.php">
  Order allow,deny
  Deny from all
</FilesMatch>

<Files ~ "^.*\.([Hh][Tt][Aa])">
    Order allow,deny
    Deny from all
    Satisfy all
</Files>

AddType x-httpd-php73 .php

I'm also using LetsEncypt - not sure if that's relevant?

Simon
  • 19
  • 8
  • `RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]` does a 301 redirect to the `http` version of your site and then the next rule redirects http URLs to https creating two redirects. You can just use a single rule to enforce both https and www. – Amit Verma Nov 12 '21 at 05:57
  • @Stephen Ostermiller sorry you are correct, I mean the home page redirects but not the sub pages – Simon Nov 12 '21 at 08:19
  • The rules you have currently are not from any of the other questions you copied, and as already pointed out redirect to http. I know you're having trouble but the answers you linked do work. One thing to remember is that browsers can cache 301 responses, so after you've changed your `.htaccess` you should also clear your browser history and/or restart your browser, otherwise you don't see the change, which I know from experience is very confusing and frustrating :-) – Don't Panic Nov 12 '21 at 08:37
  • [Here's a very simple solution](https://stackoverflow.com/questions/38069945/redirect-non-www-and-non-http-to-https), which isn't in the list you've tried, AFAICT. The accepted answer will work for a single domain, but the other answer will work for any domain. – Don't Panic Nov 12 '21 at 08:38
  • Does this answer your question? [Redirect non-www and non-http to https](https://stackoverflow.com/questions/38069945/redirect-non-www-and-non-http-to-https) – Don't Panic Nov 12 '21 at 08:38
  • The rules you are using are meant to cover every page, not just the home page. They use `(.*)` to match and capture the whole URL path and then using `$1` to insert the capture into the redirect. How are you testing this? Have you cleared your browser cache? Are you using a command line tool like https://curl.se (`curl --head https://example.co.uk`) that won't have caching problems? Do you have any other rewrite rules in your .htaccess that might be interfering? – Stephen Ostermiller Nov 12 '21 at 08:47
  • @StephenOstermiller I log via putty and use the command (curl --head https://xxx.co.uk). I'll update the original question with the entire .htaccess code. Is it okay to post the actual domain name on here? – Simon Nov 12 '21 at 11:46
  • @Don'tPanic This answer (https://stackoverflow.com/questions/38069945/redirect-non-www-and-non-http-to-https) doesn't work – Simon Nov 12 '21 at 11:55
  • @AmitVerma I have tried this `RewriteCond %{HTTPS} off [OR] RewriteCond %{HTTP_HOST} !^www\. RewriteRule ^ https://www.xxx.co.uk%{REQUEST_URI} [NC,L,R=301,NE]` which also doesn't work. Do you have any suggestions? – Simon Nov 12 '21 at 12:35
  • @Simon Those rules should definitely work. What exactly is your issue with htaccess now? – Amit Verma Nov 12 '21 at 12:40
  • @AmitVerma http://xxx.co.uk redirects to https://www.xxx.co.uk okay but none of the directories and sub-directories redirect. So http://xxx.co.uk/coffee-machines/, http://xxx.co.uk/coffee-machines/category/bean-to-cup-coffee-machines/ and http://xxx.co.uk/coffee-machines/product/delonghi-autentica-cappuccino-bean-to-cup-coffee-machine-etam29.660.sb-silver.html and so on don't redirect to their www/https equivalents. I am doing something wrong, I just can't figure out what it is. – Simon Nov 12 '21 at 12:52
  • @Simon are these real directories? Do you have any other htaccess file inside these dirs? – Amit Verma Nov 12 '21 at 13:03
  • @AmitVerma Yes, those are real directories and each top level directory has it's own .htaccess. So http://xxx.co.uk/coffee-machines/, http://xxx.co.uk/audio/, http://xxx.co.uk/kitchen-scales/, http://xxx.co.uk/mixers-blenders/ will all have their own .htaccess – Simon Nov 12 '21 at 13:05
  • 1
    @Simon okay , then the problem is that your `subdir/. htaccess` is overriding the main htaccess file. To fix this , Just put `ssl and non-www` rule at the top of your htaccess in subfolder bellow `RewriteEngine on`. – Amit Verma Nov 12 '21 at 13:08
  • 1
    @AmitVerma Thank you so much for your help. That has worked. It just didn't occur to me that the top level .htaccess wouldn't handle the redirects across the whole site. if you can add that as an answer, I'll mark it as accepted. I've been working on this for a long time so it's great to get it resolved. Thanks again. – Simon Nov 12 '21 at 13:39

1 Answers1

1

When the problem is per-directory .htaccess files, you can prevent the rules in those files from running after some rules by using the [END] flag. From the documentation:

Using the [END] flag terminates not only the current round of rewrite processing (like [L]) but also prevents any subsequent rewrite processing from occurring in per-directory (htaccess) context.

So you should be able to use [END] in your rules to make them take precedence over other rules in the subdirectories:

RewriteCond %{HTTPS} off
RewriteCond %{HTTP_HOST} !^www\.(.*)$ [NC]
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,END]

RewriteCond %{HTTPS} on
RewriteCond %{HTTP_HOST} !^www\.(.*)$ [NC]
RewriteRule ^(.*)$ https://www.%{HTTP_HOST}/$1 [R=301,END]
Stephen Ostermiller
  • 23,933
  • 14
  • 88
  • 109