Scraping iframe video from other sites through PHP

Question

I want to scrape video from other sites to my sites (e.g. from a live video site).

How can I scrape the <iframe> video from other websites? Is the process the same as that for scraping images?

$html = file_get_contents('http://website.com/');
$dom = new domDocument;
$dom->loadHTML($html);
$dom->preserveWhiteSpace = false;
$iframes = $dom->getElementsByTagName('frame');
foreach ($iframes as $iframe) {
  $pic = $iframe->getAttribute('src');
  echo '<li><frame src="'.$pic.'"';
}

So what doesn't work with your code? – Ali Oct 31 '14 at 14:58 — Ali, Oct 31 '14 at 14:58
it is printed alot or errrors, no videos scraped – Azad khan Oct 31 '14 at 15:26 — Azad khan, Oct 31 '14 at 15:26

score 0 · Answer 1 · answered May 06 '15 at 21:58

This post is a little old, but still, here's my answer:

I'd recommend you to use cURL and Xpath to scrape the site and parse the HTML data. file_get_content has some security issues and some hosts may disable it. You could do something like this:

<?php
    function scrape($URL){
        //cURL options
        $options = Array(
                    CURLOPT_RETURNTRANSFER => TRUE, //return html data in string instead of printing it out on screen
                    CURLOPT_FOLLOWLOCATION => TRUE, //follow header('Location: location');
                    CURLOPT_CONNECTTIMEOUT => 60, //max time to try to connect to page
                    CURLOPT_HEADER => FALSE, //include header
                    CURLOPT_USERAGENT => "Mozilla/5.0 (X11; Linux x86_64; rv:21.0) Gecko/20100101 Firefox/21.0", //User Agent
                    CURLOPT_URL => $URL //SET THE URL
                    );

        $ch = curl_init($URL);//initialize a cURL session
        curl_setopt_array($ch, $options);//set the cURL options
        $data = curl_exec($ch);//execute cURL (the scraping)
        curl_close($ch);//close the cURL session

        return $data;
    }

    function parse(&$data, $query, &$dom){
        $Xpath = new DOMXpath($dom); //new Xpath object associated to the domDocument
        $result = $Xpath->query($query);//run the Xpath query through the HTML
        var_dump($result);
        return $result;
    }


    //new domDocument
    $dom = new DomDocument("1.0"); 

    //Scrape and parse
    $data = scrape('http://stream-tv-series.net/2013/02/22/new-girl-s1-e6-thanksgiving/'); //scrape the website
    @$dom->loadHTML($data); //load the html data to the dom

    $XpathQuery = '//iframe'; //Your Xpath query could look something like this
    $iframes = parse($data, $XpathQuery, $dom); //parse the HTML with Xpath

    foreach($iframes as $iframe){

        $src = $iframe->getAttribute('src'); //get the src attribute
        echo '<li><iframe src="' . $src . '"></iframe></li>'; //echo the iframes
    }
?>

Here are some links that you could find useful:

cURL: http://php.net/manual/fr/book.curl.php

Xpath: http://www.w3schools.com/xpath/

There is also the DomDocument documention on php.net. I can't post the link, I don't have enough reputation.

Scraping iframe video from other sites through PHP

1 Answers1