0

I am running this website www.miswag.net which is highly dependent on Facbeook. When I share my site on Facebook, I get a "403 Forbidden", here's Facebook's debugger output when I try to scrape my site: https://developers.facebook.com/tools/debug/og/object?q=http%3A%2F%2Fwww.miswag.net

Please help figure this out.. Thanks

Ammar
  • 3
  • 3
  • Make sure Facebook scraper agent is white listed – Adam Azad Mar 27 '14 at 09:43
  • I found this in my Apache's access log: GET / HTTP/1.1" 404 - "-" "facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php). Why does it say 404? It seems Facebook is trying to access index.html which I used to keep while the site was under development and now it's removed. @AdamAzad.. Thanks – Ammar Mar 27 '14 at 11:46
  • You can check if the agent has `facebookexternalhit/1.1` and return a custom header (200 OK) `if (in_array($_SERVER['HTTP_USER_AGENT'], array( 'facebookexternalhit/1.1 (+https://www.facebook.com/externalhit_uatext.php)', 'facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)' ))) { header("HTTP/1.0 404 Not Found"); } else { echo "You're not Facebook agent '__'"; }` source: http://stackoverflow.com/a/8626828 – Adam Azad Mar 27 '14 at 13:07
  • solution: https://stackoverflow.com/a/64169793/1679903 – kinjelom Oct 02 '20 at 10:09

2 Answers2

0

Facebook's scraper doesn't need to read the full page, so they only request part of it by sending a Range header. However, your server seems to be responding incorrectly to that request and returning a 403 error code. You need to check your server and make sure it handles the range header correctly.

To see this in action, try this CURL command:

curl -H "Range: bytes=0-524287" http://www.miswag.net

This is the response I got:

<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /
on this server.</p>
<p>Additionally, a 404 Not Found
error was encountered while trying to use an ErrorDocument to handle the request.</p>
</body></html>
Waleed Abdulla
  • 1,873
  • 1
  • 14
  • 20
  • The problem is I am on a shared hosting! I was looking into my WHM and seems like I cannot go that far. I think I need consider finding a vps or something – Ammar Apr 03 '14 at 08:53
0

Compounding on @Waleed'sanswer, if you want to test right away, use Online CURL

  1. fill in www.wiswag.net
  2. ADD OPTION options, choosing --header (-H)
  3. next to it "Range: bytes=0-524287"
  4. CURL button
tony gil
  • 9,424
  • 6
  • 76
  • 100