0

I'm using Node + PhantomJS-Node to do some dynamic website crawling. The first page is opened, a link is obtained and then a second page is opened. On the second page there is a button that initiates a search which I'm having trouble clicking.

Initially I tried using jQuery.click() on the button element but when the screenshot was taken it didn't seem to have been clicked as the image on the page was not updated. I followed the solution here which is what I have implemented below but the screenshot outputted from the code below suggests that the button was not clicked.

phantom.create(function (ph) {
    ph.createPage(function (page) {
        page.open("www.mysamplepage.com", function (status) {
            page.includeJs("http://ajax.googleapis.com/ajax/libs/jquery/1.6.1/jquery.min.js", function() {

                page.evaluate(function () {
                    return $('div.Div5 a').attr('href');
                }, function (link) {
                    page.open(link, function(status) {
                        page.evaluate(function() {

                            var element = document.getElementById('mybutton');
                            var ev = document.createEvent("MouseEvent");
                            ev.initMouseEvent(
                                "click",
                                true /* bubble */, true /* cancelable */,
                                window, null,
                                0, 0, 0, 0, /* coordinates */
                                false, false, false, false, /* modifier keys */
                                0 /*left*/, null
                            );
                            element.dispatchEvent(ev);

                        }, function() {
                            setTimeout(function() {
                                console.log('status = ' + status);
                                page.render('page.png');
                                ph.exit();
                            }, 30000);
                        });


                    });
                });
            });
        });
    });
});

I'm not sure if it's related but when I executed the code that programmatically clicks the button in a Chrome browser, it's only fired once and subsequent execution of the code isn't fired.

var ev = document.createEvent("MouseEvent");
ev.initMouseEvent(
    "click",
    true /* bubble */, true /* cancelable */,
    window, null,
    0, 0, 0, 0, /* coordinates */
    false, false, false, false, /* modifier keys */
    0 /*left*/, null
);

document.getElementById('btnAddressSearch').dispatchEvent(ev); // Executed
document.getElementById('btnAddressSearch').dispatchEvent(ev); // Not executed
document.getElementById('btnAddressSearch').dispatchEvent(ev); // Not executed
Community
  • 1
  • 1
n00b
  • 5,843
  • 11
  • 52
  • 82
  • 1
    Your code looks fine. Please register to the `onConsoleMessage`, `onError`, `onResourceError`, `onResourceTimeout` events ([Example](https://gist.github.com/artjomb/4cf43d16ce50d8674fdf#file-3_phantomnodeerrors-js)). Maybe there are errors. Have you tried using the native click methods as described [here](http://stackoverflow.com/a/24026636/1816580) (don't forget to adjust for phantomjs-node)? – Artjom B. Oct 02 '15 at 13:00
  • Thanks Artjom. The logging's helped in locating an "SSL handshake failed" error, but I'm having trouble resolving this. At the moment I'm wondering whether the technology stack of Node + Phantom is not yet stable enough and considering switching to Node + Selenium, or maybe switching away from Node altogether. – n00b Oct 06 '15 at 00:47
  • Node is pretty stable and PhantomJS has many bugs and inconsistencies like any other browser. Selenium has the advantage that you can change the browser under the hood pretty easily which is not at all possible with writing your scripts for PhantomJS directly. Try to pass the `"ssl-protocol": any` and `"ignore-ssl-errors": true` parameters to phantom: http://stackoverflow.com/a/31451324 – Artjom B. Oct 06 '15 at 09:30
  • You don't need to create an event to click inside page.evaluate(). Just use document.querySelector(element).click(); For browser security reasons, Chrome blocks the dispatching of some events. I learned this the hard way when attempting to write a sendKeys() method inside the Chrome console. PhantomJS doesn't appear to have these same security rules in place, however. – GracefulCode Jan 15 '16 at 15:31
  • In any event, I don't think the click functionality is your problem. I believe the issue is with your nesting of page.evaluate(). Put some console.log statements (or console.error when inside the page.evaluate scope) to see what's going on. – GracefulCode Jan 15 '16 at 15:49

0 Answers0