0

I made a scraper using PhantomJS inside node (Node Module).

I am trying to get data from a table on the page (url). When the page loads it only displays 25 records of the table. There is a 'select' at the bottom that you can change to 'All' to see all records. How can i change the value of the select to 'All' before getting the HTML returned?

var phantom = require('phantom');

phantom.create().then(function(ph){
    ph.createPage().then(function(page){
        page.open(url).then(function(status){
            console.log(status);

            page.property('content').then(function(content){
                console.log(content);

                page.close();
                ph.exit();
            });
        });
    });
});



<select name="qs-rankings_length" aria-controls="qs-rankings" class="jcf-hidden">
    <option value="100">100</option>
    <option value="200">200</option>
    <option value="-1">All</option>
</select>
av4625
  • 303
  • 1
  • 11

1 Answers1

1

This is how I do it.

page.evaluate(function() {
    $('#select_element_selector').val('All').change();
});

I'm assuming that you have jQuery on the page.

Or, without jQuery:

page.evaluate(function() {
    document.getElementById('select').selectedIndex = 0;
    // or use document.getElementById('select').value = 'All';
}
Jonatas Nardi
  • 11
  • 3
  • I can do that if the page that im accessing has jQuery? I dont actually have to "import" jQuery for my script? (Sorry Im new to this). And i do it before the page.property........... – av4625 Oct 25 '16 at 17:30
  • I updated the answer to works without jQuery, because I don't know how to import jQuery with phantomjs. – Jonatas Nardi Oct 25 '16 at 17:38
  • Also has this - TypeError: undefined is not a constructor (evaluating 'api.page.len(event.data.pageLength).draw()') – av4625 Oct 25 '16 at 18:46
  • This is what the select tag looks like (Updated question) – av4625 Oct 25 '16 at 19:22