PHP simplexml_load_file error: Too Many Requests?

Question

I'm trying to use vimeos API to call the name of a video via its XML file. It works file if I use this code for one xml file:

    $location = "http://vimeo.com/api/v2/video/16417063.xml";
    $xml = simplexml_load_file($location);
    echo $xml->video->title;

but after I stored all the vimeo video ids in a database and used this code:

    <?php
    $seasontwo=mysql_query("SELECT s2 FROM video_ids LIMIT 1");
    while($row=mysql_fetch_array($seasontwo))
    {
    $headline=$row['s2'];
    $location = "http://vimeo.com/api/v2/video/".$headline.".xml";
    $xml = simplexml_load_file($location);
    echo $xml->video->title;
    }
    ?>

I get the error:

    Warning: simplexml_load_file(http://vimeo.com/api/v2/video/16417063.xml) [function.simplexml-load-file]: failed to open stream: HTTP request failed! HTTP/1.1 429 Too Many Requests in /home/dpnews0/public_html/tnn/wordpress/wp-content/themes/twentytwelve/content.php on line 11

    Warning: simplexml_load_file() [function.simplexml-load-file]: I/O warning : failed to load external entity "http://vimeo.com/api/v2/video/16417063.xml" in /home/dpnews0/public_html/tnn/wordpress/wp-content/themes/twentytwelve/content.php on line 11

Even though the xml file http://vimeo.com/api/v2/video/16417063.xml is in fact valid. Can anyone help me with this?

Im not sure how to fix the fact that there are too many requests — user2230755, Apr 15 '13 at 04:19
I don't even know what that means, or how it should affect it — user2230755, Apr 15 '13 at 04:23

score 0 · Answer 1 · answered Apr 15 '13 at 04:55

Too many requests is one of the http status error code,
refer here : http://en.wikipedia.org/wiki/List_of_HTTP_status_codes#4xx_Client_Error

429 Too Many Requests (RFC 6585) The user has sent too many requests in a given amount of time. Intended for use with rate limiting schemes.

The reason you getting this error is simply vimeo block your request due to unusual high activity trigger via your scripts. This is a common practice for service provider to prevent content scrapping (just like what you are trying to do).

When error 429 is throw, you won't get a valid xml response.
Therefore, you are getting the 2nd warning message of xml I/O warning.

To solve this issue, you should store the xml response (http://vimeo.com/api/v2/video/16417063.xml) locally to avoid repeatedly requests to vimeo.

One simple way is to store the xml response into disk cache.

score 0 · Answer 2 · edited Oct 07 '21 at 06:26

You are interacting with another computer you do not own and you have no control over. Because of that, other computers can change their behavior different to what you expected. That is what has happened here, your request to open the XML resulted in an error. If you don't understand the error message, no problem, you only need to deal with it.

When you interact with remote-services, you need to design for failure:

$xml = simplexml_load_file($location);

$hasLoaded = (bool) $xml;

if (!$hasLoaded) 
{
   // handle the case that the remote server could not provide valid XML
   // e.g. provide a stub, search inside a cache or what not:

   $xml = new SimpleXMLElement('<r><video><title>Unknown</title></video></r>');
}

Vimeo also provides meta-information about the HTTP Rate-Limiting it uses in the response headers ($http_response_header):

X-RateLimit-Limit: 3600
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1366033220

This shows the WAN IP used is now blocked until Mon, 15 Apr 2013 13:40:20 GMT that is ca. for one hour after I triggered a request limit (3600 requests).

I do not get a 429 Too Many Requests (RFC6585) but just a 403 Forbidden. In the end it's both because of the rate-limiting Vimeo uses.

Var-dump:

array(17) {
  [ 0] => string(22) "HTTP/1.1 403 Forbidden"
  [ 1] => string(13) "Server: nginx"
  [ 2] => string(35) "Date: Mon, 15 Apr 2013 12:49:05 GMT"
  [ 3] => string(38) "Content-Type: text/html; charset=UTF-8"
  [ 4] => string(17) "Connection: close"
  [ 5] => string(23) "X-RateLimit-Limit: 3600"
  [ 6] => string(24) "X-RateLimit-Remaining: 0"
  [ 7] => string(29) "X-RateLimit-Reset: 1366033220"
  [ 8] => string(38) "Expires: Mon, 15 Apr 2013 13:40:20 GMT"
  [ 9] => string(39) "X-UA-Compatible: IE=EmulateIE9,chrome=1"
  [10] => string(26) "X-DNS-Prefetch-Control: on"
  [11] => string(21) "Vary: Accept-Encoding"
  [12] => string(20) "X-Varnish: 366390796"
  [13] => string(6) "Age: 0"
  [14] => string(16) "Via: 1.1 varnish"
  [15] => string(18) "X-Varnish-Cache: 0"
  [16] => string(24) "X-VServer: 10.90.128.147"
}

See as well:

Examples of HTTP API Rate Limiting HTTP Response headers

PHP simplexml_load_file error: Too Many Requests?

2 Answers2