2

How to scrape all value from a DropDown menu in a website? For example in this website

There is this DropDown menu:

enter image description here

I want to get all value and save it into an array structure that's have also a link of the relative nation, for example:

Afghanistan => http://it.soccerway.com/national/afghanistan/afghan-premier-league/2015/regular-season/r32792/
Albania => http://it.soccerway.com/national/albania/super-league/20152016/regular-season/r31891/
Algeria => http://it.soccerway.com/national/algeria/ligue-1/20152016/regular-season/r31583/
...

How I can achieve this result?

Bender
  • 523
  • 6
  • 21
  • If the site you are trying to get the data from allows this kind of maneuvre, it probably has an API that you can access to retrieve this data. Did you check this possibility? – saeraphin Sep 16 '15 at 09:11
  • In the term and condition of the site allow to use the data for free usage so there is no problem. Also the site doesn't provide any API so I'm trying to create my own. – Bender Sep 16 '15 at 09:12

1 Answers1

1

This might also be helpful (using one line of PHP): jQuery load external site page

jsBin

Inspect element, right click and Copy HTML, Paste it inside your .html file.
Here's a glimpse of that HTML structure:

<ul class="list hidden">
   <li>Club Domestic (1085)</li>
   <li data-value="/national/afghanistan/a8/?ICID=SN_02_01">Afghanistan (1)</li>


var LI = document.querySelectorAll(".list li");
var result = {};

for(var i=0; i<LI.length; i++){
  var el = LI[i];
  var elData = el.dataset.value;
  if(elData) result[el.innerHTML] = elData; // Only if element has data-value attr
}

console.log( result );
Community
  • 1
  • 1
Roko C. Buljan
  • 196,159
  • 39
  • 305
  • 313
  • Interesting solution. Just one thing: it's possible download all the html page and access on this specific element? Because the site could be change the value in the Menu.. – Bender Sep 16 '15 at 09:26
  • Yes, using PHP and `file_get_contents` function. So basically you scrate that page's content into yoru web domain (so that you don't have any cross-site AJAX issues), than you get the desired element easily from that page once it's on your server. – Roko C. Buljan Sep 16 '15 at 09:29
  • So the best way to scrape the site content is using PHP? Because I've saw that the site structure, specially the table, is in json format. – Bender Sep 16 '15 at 09:30
  • @Bender see this answer of mine: http://stackoverflow.com/questions/14999573/jquery-load-external-site-page – Roko C. Buljan Sep 16 '15 at 09:30
  • @Bender same stuff. Get the path to that JSON file and use PHP the same way. The important thing is that if they do not have any API or restrict JSONP - than you cannot simply AJAX an external website for content without using PHP or another Server-side language. – Roko C. Buljan Sep 16 '15 at 09:32
  • Okay, but in the other code questions you linked there's the ContentUri variable, in the #element I should insert the name of the class of the DropDown menu? like: – Bender Sep 16 '15 at 09:34
  • @Bender now I've seen that there's two things, both the ` – Roko C. Buljan Sep 16 '15 at 09:38