PhantomJs: Looking for a really base way to compile a form, do a click and log the result

Question

I'm trying for the first time PhantomJs. My goal is simple simple simple page automation.

I only need to - load a web page, let's say www.google.com - fill the query term - simply (really, only this) console.log of search result page

I'm having some problem because all of tutorials I've found starts with too complicated task.

What follows is my personal step by step tutorial, but this 'ends' with a question: "how to log the content of page after submitting ?"

The first STEP I've done is:

var page = require('webpage').create();
page.open('https://www.google.it', function () {
    console.log (page.content);
    phantom.exit()
});

And this works.

Then I inject jquery

var page = require('webpage').create();
page.open('https://www.google.it', function () {

    page.includeJs("http://code.jquery.com/jquery-1.10.1.min.js", function() {

        console.log (page.content);
        phantom.exit()
    });

});

An that's working.

Now I'm trying to fill the search box. I must fill up this field

<input autocomplete="off" class="lst" value="" title="Cerca con Google" maxlength="2048" name="q" size="57" style="...[omitted]...">

I'm doing this:

var page = require('webpage').create();
page.open('https://www.google.it', function () {

    page.includeJs("http://code.jquery.com/jquery-1.10.1.min.js", function() {

        console.log (page.content);
        var value = page.evaluate(function() {
            $('input[name="q"]').val("Daduu");
            return $('input[name="q"]').val();
        });
        console.log ("search term: " + value);
        phantom.exit()
    });

});

And it's working. Now I need to 'click' on submit button

<input class="lsb" value="Cerca con Google" name="btnG" type="submit">

Doing this:

var page = require('webpage').create();
page.open('https://www.google.it', function () {

    page.includeJs("http://code.jquery.com/jquery-1.10.1.min.js", function() {


        var value = page.evaluate(function() {
            $('input[name="q"]').val("Daduu");
            $('input[name="btnG"]').trigger("click");
            return $('input[name="q"]').val();
        });
        // console.log (page.content);
        page.render("google.png");
        phantom.exit()
    });

});

When I see the image ( just to see rendered results in a human-way), I see the google.it page with filled 'q' field.

The questions are:

**Is really the page loading [is it Google searching] ? **

"How to detect page change to console.log or page.render the content ?"

EDIT: I see this: Phantomjs - How to populate a form, submit and get the results?

But this is NOT what I need. I want to detect when page loading is complete, not to 'settimeout' to render page.

EDIT 2: I see this: PhantomJS: submit a form

But this is too complicated, I've not understood how this works.

Probably it's needed to wrap all around a setTimeout, but, because of using js objects, i'm looking for a way to work event-driven

onInitialPageLoaded (
  fillForm
  click
)

onSearchResultLoaded (
  render or log content of page
)

But I cannot understand if is it possible in PhantomJs, and, at all, how to realize it !

It very much depends on site's internals. Almost everything is possible but sometimes you pretty much have to know how certain site works. Some of contents is loaded via ajax so you won't catch events with hooks, so the simplest way here is using setTimeout. Using one you will see that search button click event did not start loading results for some reason. To find that reason you have to experiment and read google's code, so I hardly think there are simple answers to your questions. — igor, Jun 07 '13 at 10:24

score 1 · Accepted Answer · edited Jun 07 '13 at 13:35

PhantomJS itself is not a test framework. There are projects which are built on top of PhantomJS to provide convenient high-level functionality for testing purposes such as CasperJs.

Let's search "phantomjs" with casperJs.

var links = [];
var casper = require('casper').create();

function getLinks() {
    var links = document.querySelectorAll('h3.r a');
    return Array.prototype.map.call(links, function(e) {
        return e.getAttribute('href')
    });
}

casper.start(function() {
    // now search for 'phantomjs' by filling the form again
    this.fill('form[action="/search"]', { q: 'phantomjs' }, true);
});

casper.then(function() {
    // aggregate results for the 'phantomjs' search
    links = links.concat(this.evaluate(getLinks));
});

casper.run(function() {
    // echo results in some pretty fashion
    this.echo(links.length + ' links found:');
    this.echo(' - ' + links.join('\n - ')).exit();
});

From Ariya Hidayat, creator of PhantomJS

In case you haven’t seen CasperJS yet, go and take a look, it’s an extremely useful companion to PhantomJS.

Thanks, but my question was related ONLY to phantomJs – realtebo Jun 07 '13 at 14:01 — realtebo, Jun 07 '13 at 14:01

PhantomJs: Looking for a really base way to compile a form, do a click and log the result

1 Answers1