PHP: Reduce load of function that gets content from external resource</a></h1> </div> <div class="grid fw-wrap pb8 mb16 bb bc-black-075"> <div class="grid--cell ws-nowrap mr16 mb8" title="2016-01-12 19:07:53Z"> <span class="fc-light mr2">Asked</span> <time itemprop="dateCreated" datetime="2013-12-27T23:50:32.940" class="fromnow">Dec 27 '13 at 23:50</time> </div> <div class="grid--cell ws-nowrap mr16 mb8"> <span class="fc-light mr2">Active</span> <time class="fromnow" title="2013-12-28T13:57:38.730" datetime="2013-12-28T13:57:38.730">Dec 28 '13 at 13:57</a> </div> <div class="grid--cell ws-nowrap mb8" title="Viewed 271 times"> <span class="fc-light mr2">Viewed</span> 271 times </div> </div> <div id="mainbar" role="main" aria-label="questions and answers"> <div id="question" class="question" data-questionid="20809416" data-ownerid="1342772" data-score="0"> <div class="post-layout"> <div class="votecell post-layout--left"> <div class="js-voting-container grid jc-center fd-column ai-stretch gs4 fc-black-200" data-post-id="20809416"> <button class="js-vote-up-btn grid--cell s-btn s-btnunset c-pointer"><svg aria-hidden="true" class="m0 svg-icon iconArrowUpLg" width="36" height="36" viewBox="0 0 36 36"><path d="M2 26h32L18 10 2 26z"></path></svg></button> <div class="js-vote-count grid--cell fc-black-500 fs-title grid fd-column ai-center" itemprop="upvoteCount" data-value="0">0</div> <button class="js-bookmark-btn s-btn s-btnunset c-pointer py4"> <svg aria-hidden="true" class="svg-icon iconBookmark" width="18" height="18" viewBox="0 0 18 18"><path d="M6 1a2 2 0 00-2 2v14l5-4 5 4V3a2 2 0 00-2-2H6zm3.9 3.83h2.9l-2.35 1.7.9 2.77L9 7.59l-2.35 1.7.9-2.76-2.35-1.7h2.9L9 2.06l.9 2.77z"></path></svg> <div class="js-bookmark-count mt4" data-value=""></div> </button> </div> </div> <div class="postcell post-layout--right"> <div class="s-prose js-post-body" itemprop="text"><p>I created a function that checks if the <code><title></code> tag of an external page contains specific words (between the others of the title). If check is positive it should echo the (whole) page <code><title></code>.</p> <pre><code><?php function file_get_contents_curl($url) { $ch = curl_init(); curl_setopt($ch, CURLOPT_HEADER, 0); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); $data = curl_exec($ch); curl_close($ch); return $data; } $html = file_get_contents_curl("http://www.lastfm.it/user/lorenzone92/now"); $doc = new DOMDocument(); @$doc->loadHTML($html); $nodes = $doc->getElementsByTagName('title'); $title = $nodes->item(0)->nodeValue; if (strpos($title,'in ascolto') !== false) { echo "$title". '<br>'; } ?> </code></pre> <p>It is working fine. My concern is about memory consumption and server load. The problem is that I cannot cache the $html because it's a live thing.. any idea? Do I need to grab the whole page to just access the <code><title></code>? Other methods instead of cURL and file_get_contents to reduce server load? Or I'm just overconcerned..? :)</p> <p>Note: Don't worry about PHP version ( no limits, I'm on my VPS which has PHP 5.5.7 installed :D ).</p></div> <div class="mt24 mb12"> <div class="post-taglist grid gs4 gsy fd-column"> <div class="grid ps-relative"> <a href="../../questions/tagged/php" class="post-tag js-gps-track" title="show questions tagged 'php'" rel="tag">php</a> <a href="../../questions/tagged/optimization" class="post-tag js-gps-track" title="show questions tagged 'optimization'" rel="tag">optimization</a> <a href="../../questions/tagged/curl" class="post-tag js-gps-track" title="show questions tagged 'curl'" rel="tag">curl</a> <a href="../../questions/tagged/memory-consumption" class="post-tag js-gps-track" title="show questions tagged 'memory-consumption'" rel="tag">memory-consumption</a> <a href="../../questions/tagged/server-load" class="post-tag js-gps-track" title="show questions tagged 'server-load'" rel="tag">server-load</a> </div> </div> </div> <div class="mb0"> <div class="mt16 grid gs8 gsy fw-wrap jc-end ai-start pt4 mb16"> <div class="grid--cell mr16 fl1 w96"></div> <div class="post-signature grid--cell"> <div class="user-info "> <div class="user-action-time">edited <span title="2013-12-28T13:57:38.730" class="relativetime">Dec 28 '13 at 13:57</span></div> <div class="user-gravatar32"></div> <div class="user-details" itemprop="author" itemscope="" itemtype="http://schema.org/Person"> <span class="d-none" itemprop="name">MultiformeIngegno</span> <div class="-flair"></div> </div> </div> </div> <div class="post-signature owner grid--cell"> <div class="s-user-card s-user-card"> <time class="s-user-card--time" datetime="asked Dec 27 '13 at 23:50">asked Dec 27 '13 at 23:50</time> <a href="../../users/1342772/multiformeingegno" class="s-avatar s-avatar32 s-user-card--avatar"> <img class="s-avatar--image" src="../../users/profiles/1342772.webp" data-jdenticon-width="32" data-jdenticon-height="32" data-jdenticon-value="MultiformeIngegno" /> </a> <div class="s-user-card--info"> <a href="../../users/1342772/multiformeingegno" class="s-user-card--link">MultiformeIngegno</a> <ul class="s-user-card--awards"> <li class="s-user-card--rep" title="reputation score">6,959</li> <li class="s-award-bling s-award-blinggold" title="15 gold badges">15</li> <li class="s-award-bling s-award-blingsilver" title="60 silver badges">60</li> <li class="s-award-bling s-award-blingbronze" title="119 bronze badges">119</li> </ul> </div> </div> </div> </div> </div> </div> <div class="post-layout--right js-post-comments-component"> <div id="comments-20809416" class="comments js-comments-container bt bc-black-075 mt12 " data-post-id="20809416" data-min-length="15"> <ul class="comments-list js-comments-list" data-remaining-comments-count="0" data-canpost="false" data-cansee="true" data-comments-unavailable="false" data-addlink-disabled="true"> <li id="comment-31200346" class="comment js-comment " data-comment-id="31200346" data-comment-owner-id="2588457" data-comment-score="0"> <div class="js-comment-actions comment-actions"> <div class="comment-score js-comment-edit-hide"> </div> </div> <div class="comment-text js-comment-text-and-form"> <a name="comment31200346_20809416"></a> <div class="comment-body js-comment-edit-hide"> <span class="comment-copy">I'm not sure, but I think there is no solution to your problem.</span> – <a href="../../users/2588457/eisa-adil" title="1,743 reputation" class="comment-user ">Eisa Adil</a> <span class="comment-date" dir="ltr"><a class="comment-link" href="../../questions/20809416/php-reduce-load-of-function-that-gets-title-content-from-external-resource#comment31200346_20809416"><span title="2013-12-27T23:52:35.520 License: CC BY-SA 3.0" class="relativetime-clean">Dec 27 '13 at 23:52</span></a></span> </div> </div> </li> </ul> </div> </div> </div> </div> <div id="answers"> <a name="tab-top"></a> <div id="answers-header"> <div class="answers-subheader grid ai-center mb8"> <div class="grid--cell fl1"> <h2 class="mb0" data-answercount="9">3 Answers<span style="display:none;" itemprop="answerCount">3</span></h2> </div> </div> </div> <a name="20809453"></a> <div id="answer-20809453" class="answer " data-answerid="20809453" data-ownerid="3139567" data-score="0" itemprop="suggestedAnswer" itemscope="" itemtype="https://schema.org/Answer"> <div class="post-layout"> <div class="votecell post-layout--left"> <div class="js-voting-container grid jc-center fd-column ai-stretch gs4 fc-black-200" data-post-id="20809453"> <button class="js-vote-up-btn grid--cell s-btn s-btnunset c-pointer"><svg aria-hidden="true" class="m0 svg-icon iconArrowUpLg" width="36" height="36" viewBox="0 0 36 36"><path d="M2 26h32L18 10 2 26z"></path></svg></button> <div class="js-vote-count grid--cell fc-black-500 fs-title grid fd-column ai-center" itemprop="upvoteCount" data-value="0">0</div> </div> </div> <div class="postcell post-layout--right"> <div class="s-prose js-post-body" itemprop="text"><p>I do not know if it's helpful... but this other question (that seem related to yours) seem to have a lot of answers... here the link</p> <p><a href="../../questions/4348912/get-title-of-website-via-link">Get title of website via link</a></p></div> <div class="mb0"> <div class="mt16 grid gs8 gsy fw-wrap jc-end ai-start pt4 mb16"> <div class="grid--cell mr16 fl1 w96"></div> <div class="post-signature grid--cell"> <div class="s-user-card s-user-card"> <time class="s-user-card--time" datetime="edited May 23 '17 at 12:03">edited May 23 '17 at 12:03</time> <a href="../../users/-1/community" class="s-avatar s-avatar32 s-user-card--avatar"> <img class="s-avatar--image" src="../../users/profiles/-1.webp" data-jdenticon-width="32" data-jdenticon-height="32" data-jdenticon-value="Community" /> </a> <div class="s-user-card--info"> <a href="../../users/-1/community" class="s-user-card--link">Community</a> <ul class="s-user-card--awards"> <li class="s-user-card--rep" title="reputation score">1</li> <li class="s-award-bling s-award-blingsilver" title="1 silver badges">1</li> </ul> </div> </div> </div> <div class="post-signature grid--cell"> <div class="s-user-card s-user-card"> <time class="s-user-card--time" datetime="answered Dec 27 '13 at 23:54">answered Dec 27 '13 at 23:54</time> <a href="../../users/3139567/goikiu" class="s-avatar s-avatar32 s-user-card--avatar"> <img class="s-avatar--image" src="../../users/profiles/3139567.webp" data-jdenticon-width="32" data-jdenticon-height="32" data-jdenticon-value="Goikiu" /> </a> <div class="s-user-card--info"> <a href="../../users/3139567/goikiu" class="s-user-card--link">Goikiu</a> <ul class="s-user-card--awards"> <li class="s-user-card--rep" title="reputation score">574</li> <li class="s-award-bling s-award-blingsilver" title="3 silver badges">3</li> <li class="s-award-bling s-award-blingbronze" title="12 bronze badges">12</li> </ul> </div> </div> </div> </div> </div> </div> <div class="post-layout--right js-post-comments-component"> <div id="comments-20809453" class="comments js-comments-container bt bc-black-075 mt12 " data-post-id="20809453" data-min-length="15"> <ul class="comments-list js-comments-list" data-remaining-comments-count="0" data-canpost="false" data-cansee="true" data-comments-unavailable="false" data-addlink-disabled="true"> <li id="comment-31200392" class="comment js-comment " data-comment-id="31200392" data-comment-owner-id="1342772" data-comment-score="0"> <div class="js-comment-actions comment-actions"> <div class="comment-score js-comment-edit-hide"> </div> </div> <div class="comment-text js-comment-text-and-form"> <a name="comment31200392_20809453"></a> <div class="comment-body js-comment-edit-hide"> <span class="comment-copy">Links to other questions should be in comments</span> – <a href="../../users/1342772/multiformeingegno" title="6,959 reputation" class="comment-user owner">MultiformeIngegno</a> <span class="comment-date" dir="ltr"><a class="comment-link" href="../../questions/20809416/php-reduce-load-of-function-that-gets-title-content-from-external-resource#comment31200392_20809453"><span title="2013-12-27T23:55:46.353 License: CC BY-SA 3.0" class="relativetime-clean">Dec 27 '13 at 23:55</span></a></span> </div> </div> </li> <li id="comment-31200401" class="comment js-comment " data-comment-id="31200401" data-comment-owner-id="3139567" data-comment-score="0"> <div class="js-comment-actions comment-actions"> <div class="comment-score js-comment-edit-hide"> </div> </div> <div class="comment-text js-comment-text-and-form"> <a name="comment31200401_20809453"></a> <div class="comment-body js-comment-edit-hide"> <span class="comment-copy">sorry, haven't read about that yet ._. i won't do it another time.</span> – <a href="../../users/3139567/goikiu" title="574 reputation" class="comment-user ">Goikiu</a> <span class="comment-date" dir="ltr"><a class="comment-link" href="../../questions/20809416/php-reduce-load-of-function-that-gets-title-content-from-external-resource#comment31200401_20809453"><span title="2013-12-27T23:56:18.463 License: CC BY-SA 3.0" class="relativetime-clean">Dec 27 '13 at 23:56</span></a></span> </div> </div> </li> </ul> </div> </div> </div> </div> <a name="20809461"></a> <div id="answer-20809461" class="answer " data-answerid="20809461" data-ownerid="913619" data-score="0" itemprop="suggestedAnswer" itemscope="" itemtype="https://schema.org/Answer"> <div class="post-layout"> <div class="votecell post-layout--left"> <div class="js-voting-container grid jc-center fd-column ai-stretch gs4 fc-black-200" data-post-id="20809461"> <button class="js-vote-up-btn grid--cell s-btn s-btnunset c-pointer"><svg aria-hidden="true" class="m0 svg-icon iconArrowUpLg" width="36" height="36" viewBox="0 0 36 36"><path d="M2 26h32L18 10 2 26z"></path></svg></button> <div class="js-vote-count grid--cell fc-black-500 fs-title grid fd-column ai-center" itemprop="upvoteCount" data-value="0">0</div> </div> </div> <div class="postcell post-layout--right"> <div class="s-prose js-post-body" itemprop="text"><p>I guess you have to load the whole page. You don't know on what position and how long is the title tag so you can't read e.g. the first 1000 characters. I don't know how many pages you try to load at the same time, but you don't load the whole media data like images and css files, so your parsed HTML code should not be too large.</p></div> <div class="mb0"> <div class="mt16 grid gs8 gsy fw-wrap jc-end ai-start pt4 mb16"> <div class="grid--cell mr16 fl1 w96"></div> <div class="post-signature grid--cell"> <div class="s-user-card s-user-card"> <time class="s-user-card--time" datetime="answered Dec 27 '13 at 23:55">answered Dec 27 '13 at 23:55</time> <a href="../../users/913619/imx" class="s-avatar s-avatar32 s-user-card--avatar"> <img class="s-avatar--image" src="../../users/profiles/913619.webp" data-jdenticon-width="32" data-jdenticon-height="32" data-jdenticon-value="iMx" /> </a> <div class="s-user-card--info"> <a href="../../users/913619/imx" class="s-user-card--link">iMx</a> <ul class="s-user-card--awards"> <li class="s-user-card--rep" title="reputation score">846</li> <li class="s-award-bling s-award-blinggold" title="1 gold badge">1</li> <li class="s-award-bling s-award-blingsilver" title="9 silver badge">9</li> <li class="s-award-bling s-award-blingbronze" title="23 bronze badge">23</li> </ul> </div> </div> </div> </div> </div> </div> <div class="post-layout--right js-post-comments-component"> </div> </div> </div> <a name="20809561"></a> <div id="answer-20809561" class="answer " data-answerid="20809561" data-ownerid="548020" data-score="0" itemprop="suggestedAnswer" itemscope="" itemtype="https://schema.org/Answer"> <div class="post-layout"> <div class="votecell post-layout--left"> <div class="js-voting-container grid jc-center fd-column ai-stretch gs4 fc-black-200" data-post-id="20809561"> <button class="js-vote-up-btn grid--cell s-btn s-btnunset c-pointer"><svg aria-hidden="true" class="m0 svg-icon iconArrowUpLg" width="36" height="36" viewBox="0 0 36 36"><path d="M2 26h32L18 10 2 26z"></path></svg></button> <div class="js-vote-count grid--cell fc-black-500 fs-title grid fd-column ai-center" itemprop="upvoteCount" data-value="0">0</div> </div> </div> <div class="postcell post-layout--right"> <div class="s-prose js-post-body" itemprop="text"><p>I simple way to load only a part of a site is the <a class="external-link" href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.35" rel="nofollow"><code>Range</code></a> header:</p> <pre><code>Range: bytes=0-499 </code></pre> <p>If the server supports the Range header, it only returns the first 500 bytes. Unfortunately, this breaks the mark-up of the page which might result in errors when using <code>DOMDocument</code>. On the other hand, using <code>DOMDocument</code> is probably not the best idea when you only need the content of one HTML element. I recommend using a simple regex or basic string functions.</p></div> <div class="mb0"> <div class="mt16 grid gs8 gsy fw-wrap jc-end ai-start pt4 mb16"> <div class="grid--cell mr16 fl1 w96"></div> <div class="post-signature grid--cell"> <div class="s-user-card s-user-card"> <time class="s-user-card--time" datetime="answered Dec 28 '13 at 00:08">answered Dec 28 '13 at 00:08</time> <a href="../../users/548020/codezombie" class="s-avatar s-avatar32 s-user-card--avatar"> <img class="s-avatar--image" src="../../users/profiles/548020.webp" data-jdenticon-width="32" data-jdenticon-height="32" data-jdenticon-value="CodeZombie" /> </a> <div class="s-user-card--info"> <a href="../../users/548020/codezombie" class="s-user-card--link">CodeZombie</a> <ul class="s-user-card--awards"> <li class="s-user-card--rep" title="reputation score">5,367</li> <li class="s-award-bling s-award-blinggold" title="3 gold badges">3</li> <li class="s-award-bling s-award-blingsilver" title="30 silver badges">30</li> <li class="s-award-bling s-award-blingbronze" title="37 bronze badges">37</li> </ul> </div> </div> </div> </div> </div> </div> <div class="post-layout--right js-post-comments-component"> <div id="comments-20809561" class="comments js-comments-container bt bc-black-075 mt12 " data-post-id="20809561" data-min-length="15"> <ul class="comments-list js-comments-list" data-remaining-comments-count="0" data-canpost="false" data-cansee="true" data-comments-unavailable="false" data-addlink-disabled="true"> <li id="comment-31211368" class="comment js-comment " data-comment-id="31211368" data-comment-owner-id="1342772" data-comment-score="0"> <div class="js-comment-actions comment-actions"> <div class="comment-score js-comment-edit-hide"> </div> </div> <div class="comment-text js-comment-text-and-form"> <a name="comment31211368_20809561"></a> <div class="comment-body js-comment-edit-hide"> <span class="comment-copy">Thanks for the reply! Why would it result in errors with DOMDocument?</span> – <a href="../../users/1342772/multiformeingegno" title="6,959 reputation" class="comment-user owner">MultiformeIngegno</a> <span class="comment-date" dir="ltr"><a class="comment-link" href="../../questions/20809416/php-reduce-load-of-function-that-gets-title-content-from-external-resource#comment31211368_20809561"><span title="2013-12-28T13:58:46.720 License: CC BY-SA 3.0" class="relativetime-clean">Dec 28 '13 at 13:58</span></a></span> </div> </div> </li> </ul> </div> </div> </div> </div> </div> </div> </div> </div> <script src="../../static/js/stack-icons.js"></script> <script src="../../static/js/fromnow.js"></script> </body> </html>