403 Error with all URLs end only with .com

Question

In our website we have domains lookup, made on laravel, we are facing 403 error in Google Webmaster only those URLs which end .com

Example :

https://v2.domainsanalytics.com/allheartweb.com

Result in webmaster

Failed: Blocked due to access forbidden (403)

Another URLs have no issue

Example :

https://v2.domainsanalytics.com/allheartweb.net

Result in webmaster

Time
18 Aug 2022, 20:52:53
Crawled as
Googlebot smartphone
Crawl allowed?
Yes
Page fetch
Successful
Indexing allowed?
Yes

My htaccess file is

<IfModule mod_rewrite.c>
    <IfModule mod_negotiation.c>
        Options -MultiViews -Indexes
    </IfModule>

    RewriteEngine On

       # Send Requests To Front Controller...
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_FILENAME} !-f
    RewriteRule ^ index.php [L]
    
    # Handle Authorization Header
    RewriteCond %{HTTP:Authorization} .
    RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

    # Redirect Trailing Slashes If Not A Folder...
    RewriteCond %{REQUEST_FILENAME} !-d
    RewriteCond %{REQUEST_URI} (.+)/$
    

</IfModule>

# php -- BEGIN cPanel-generated handler, do not edit
# Set the “ea-php74” package as the default “PHP” programming language.
<IfModule mime_module>
  AddHandler application/x-httpd-ea-php74 .php .php7 .phtml
</IfModule>
# php -- END cPanel-generated handler, do not edit

Do you only see the crawl error in GSC or are you able to reproduce the 403? _Aside:_ Your mod_rewrite directives are in the wrong order and the last rule is incomplete. — MrWhite, Aug 19 '22 at 01:24
The result in the web browser is the same btw., also a 403 status code. But it also returns the HTML with the actual API response about the requested domain - which is a pretty clear indicator, that this is not due to your rewriting, the request obviously appears to have reached your script. — CBroe, Aug 19 '22 at 06:19
You asked effectively the same question back on 21-May-2022: https://stackoverflow.com/questions/72327161/403-error-when-test-in-schema-testing-tool - At the time is was suggested that `.htaccess` was unlikely to be the problem (since your pages appear to be served successfully). — MrWhite, Aug 19 '22 at 11:29

score 2 · Accepted Answer · answered Aug 19 '22 at 10:59

2

.com as a file extension is a Windows Command file. Those executable scripts are often used to distribute malware. Your URL that ends in .com looks like a malware download. There is likely a rule on some web server, CDN, or firewall that is blocking those URLs.

I'd suggest adding a trailing slash to your URLs:

https://v2.domainsanalytics.com/allheartweb.com/
https://v2.domainsanalytics.com/allheartweb.net/

For reference see When should I use a trailing slash in my URL?

answered Aug 19 '22 at 10:59

Stephen Ostermiller

23,933
14
88
109

Although (bizarrely) the URL is not actually "blocked". The page appears to be served successfully, except with a 403 status. Ordinarily I would have said this was an error with the application? – MrWhite Aug 19 '22 at 11:38
I suspect that whatever is setting the status is doing so based on the file extension. Since it is hard to locate that rule, changing the URL would at the very least be a good test. – Stephen Ostermiller Aug 19 '22 at 12:13

403 Error with all URLs end only with .com

1 Answers1