2

I am trying to run my URL through Facebook's Debugger to see if I have setup the correct meta tags, and also, to see that stuff works in general.

When I put my URL in, I get a 403 Forbidden in return.

The URL is to a subdomain, and the domain itself also returns 403 Forbidden.

Reading another post Facebook links to my site resolve as 403 forbidden, I think that my host might have setup a block or something, however, I can't seem to recreate the block from my computer.

I have been trying to spoof the referrer and the user agent, but without luck. Before calling my host, what could I have missed?

If it's of relevance the URL in question is: http://distorpia.dadiugames.dk/

phwd
  • 19,975
  • 5
  • 50
  • 78
Mads Ohm Larsen
  • 3,315
  • 3
  • 20
  • 22

2 Answers2

2

The command I use to test this issue is curl -i -H 'Accept: */*' -H 'Range: bytes=0-40960' -H 'Connection: close' -A 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' http://whatever.com - this includes some of the headers sent by the Facebook crawler that some hosts have problems with

Looks like your hosting setup can't handle one/some of those headers properly - I replicated the 403 error:

For your URL, this returned:

HTTP/1.1 403 Forbidden
Date: Tue, 13 Dec 2011 20:54:07 GMT
Server: Apache/2.2.6 mod_auth_kerb/5.3 PHP/5.2.17 mod_fcgid/2.3.6
Accept-Ranges: bytes
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<title>Parallels H-Sphere</title>

{{REMOVED THIS PART}}

    <h2>Error 403: Forbidden</h2>

<p>You don't have permissions to access this page.
This usually means one of the following:</p>
<ul>
<li>this file and directory permissions make them unavailable from the Internet.</li>
<li>.htaccess contains instructions that prevent public access to this file or directory.</li>
</ul>
<p>Please check file and directory permissions and .htaccess configuration if you are able to do this.
Otherwise, request your webmaster to grant you access.</p>
Igy
  • 43,710
  • 8
  • 89
  • 115
  • Thanks. It works fine with all the above headers, but the "Range" one. Are you sure Facebook sends "Range: bytes=0-40960" and not "Content-Range: bytes=0-40960"? This works. I'll be calling my host in the morning for an explanation. How I wish I had access to the logs :) – Mads Ohm Larsen Dec 13 '11 at 23:43
  • I'm nearly sure it's 'Range' based on previous testing, i'll see if i can get a definite answer from the crawler code – Igy Dec 14 '11 at 08:55
  • Yep, definitely 'Range', 'bytes=0-40960' - checked the code of the crawler itself – Igy Dec 14 '11 at 08:57
0

Facebook & Facebook Debugger sends:

curl -i -H 'Accept-Encoding: deflate, gzip' -H 'Accept: */*' -H 'Range: bytes=0-524287' -H 'User-Agent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' https://your.website/something

The problem may be in the owasp-modsecurity-crs rule #958291, which gives false-positive result: https://github.com/SpiderLabs/owasp-modsecurity-crs/issues/173

Check it by sending request with header Range: bytes=524287 (in place of Range: bytes=0-524287):

curl -i -H 'Accept-Encoding: deflate, gzip' -H 'Accept: */*' -H 'Range: bytes=524287' -H 'User-Agent: facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' https://your.website/something

Solution

kinjelom
  • 6,105
  • 3
  • 35
  • 61