simple html dom url length error

Question

<?php
include('../simple_html_dom.php');

$fname = "http://www.myurl.com";

$html = file_get_html($fname);

$divs = $html->find('h6');
foreach($divs as $element)
{
 $title = $element->find('a', 0)->plaintext;
 echo $title.'<br>';
}
echo '<br>';
?>

I got this error:

"failed to open stream: HTTP request failed! HTTP/1.1 500 Internal Server Error in......."

My url is very length, its actual length is 750 characters. if I use wget it show "file name too long"

How can I fix it? I need it to work with simple dom

Then that's probably the server refusing to serve your request because of User-Agent or something. Do you control the server for this URL? — rixo, Jun 01 '13 at 13:30
That's strange. I tried with both Chrome and FF and your link results in a 500 error (whereas my link works in both of them). — gkalpak, Jun 01 '13 at 13:53

score 2 · Accepted Answer · edited May 23 '17 at 12:11

750 characters is OK for an URL length. The practical limit most often used is 2000 chars, which is the limit in older IEs.

You should try to emulate a web browser making the request. See this other question.

Edit: Using CURL with your code

<?php

// include is not a function, don't use parens (also use require instead)
require '../simple_html_dom.php';

$fname = "http://www.myurl.com";

$agent= 'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.0.3705; .NET CLR 1.1.4322)';

$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// don't want to polute your output
//curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_URL, $fname);
$result=curl_exec($ch);

$html = new simple_html_dom();
$html->load($result);

$divs = $html->find('h6');
foreach($divs as $element)
{
 $title = $element->find('a', 0)->plaintext;
 echo $title.'<br>';
}
echo '<br>';

But youtubeonfire.com will serve will `file_get_html`. So no need to try this solution... — rixo, Jun 01 '13 at 14:01
Thank you ... Problem is... that url changed link with every ip... it work fine with my localhost.. not work same link with my another apache server. thank u — user2248737, Jun 01 '13 at 14:11

score 0 · Answer 2 · answered Jun 01 '13 at 13:50

The URL length is fine. The link is probably broken or has expired. I tried with the link shown below and the results seem fine:

<?php
include("simple_html_dom.php");

$fname = "http://www.youtubeonfire.com/?genre=0&language=0&next_token=rO0ABXNyACdjb20uYW1hem9uLnNkcy5RdWVyeVByb2Nlc3Nvci5Nb3JlVG9rZW7racXLnINNqwMA%0AC0kAFGluaXRpYWxDb25qdW5jdEluZGV4WgAOaXNQYWdlQm91bmRhcnlKAAxsYXN0RW50aXR5SURa%0AAApscnFFbmFibGVkSQAPcXVlcnlDb21wbGV4aXR5SgATcXVlcnlTdHJpbmdDaGVja3N1bUkACnVu%0AaW9uSW5kZXhaAA11c2VRdWVyeUluZGV4TAANY29uc2lzdGVudExTTnQAEkxqYXZhL2xhbmcvU3Ry%0AaW5nO0wAEmxhc3RBdHRyaWJ1dGVWYWx1ZXEAfgABTAAJc29ydE9yZGVydAAvTGNvbS9hbWF6b24v%0Ac2RzL1F1ZXJ5UHJvY2Vzc29yL1F1ZXJ5JFNvcnRPcmRlcjt4cAAAAAEAAAAAAAABds0AAAAAAQAA%0AAAC71ED7AAAAAAFwdAAQMDAwMDAwMDAwMDAwMjAxM35yAC1jb20uYW1hem9uLnNkcy5RdWVyeVBy%0Ab2Nlc3Nvci5RdWVyeSRTb3J0T3JkZXIAAAAAAAAAABIAAHhyAA5qYXZhLmxhbmcuRW51bQAAAAAA%0AAAAAEgAAeHB0AApERVNDRU5ESU5HeA%3D%3D&sort=2";

$html = file_get_html($fname);

$divs = $html->find("h6");
foreach($divs as $element) {
    $title = $element->find("a", 0)->plaintext;
    echo($title . "<br />");
}
echo("<br />");

Output:

Spider (2013)
500 MPH STORM 2013 HD
Van Diemans Land (Action,Adventure,20...
Good Agent is A Bad Agent (Full HQ En...
Employee of the Month (Full HQ Englis...
The Croods (2013)
GIRLFRIENDS - 2013
Boys Are Pigs-2013
The Patriot -2013
My Daughter&#x27;s Secret -2013
Dead on Arrival [2013]
Flght 2013XViD1
Samsung Galaxy S4 Presentation UNPACK...
Affinity 2013
Golden Globe Awards 2013: Full Show
Parker-2013
Hells&#x27; Kitchen-  New Action Movie 2013
ALIENS [2013]
7 Nights Of Darkness -2013
Hansel And Gretel 2013
The Collection (2012)
Mac And Devin Go To High School 2012
Red Dawn (2012)
Hijacked -2012
Bending The Rules -2012
Inside -2012
VAMPIRELAND-2012
Dead Mine -2012
Devil Seed-2012
Kill Em All -2012
One In The Chamber -2012
The Forger - 2012
Dark Desire -2012
A Common Man -2012 .
The Helpers -2012
Red Dawn- 2012 720p

So, fix the problem with the URL and everything will work just fine !

Double-check your link. Are you sure it is working (right now) from a browser ? — gkalpak, Jun 01 '13 at 14:02

score 0 · Answer 3 · edited Jun 02 '13 at 03:39

You say your URL is working in your browser, while all of us here receive a 500 error, like your script does.

The site probably check the token in the URL against the IP and possibly other headers of the request. So you need to find a way to get a tokenized URL from your PHP script.

For that, you need to first download main page from you PHP script, then find the next link's URL and use this one in your script.

simple html dom url length error

3 Answers3