Phantomjs need the page source in spite of JS errors

Question

I have the following RequestURL.js file.

var webPage = require('webpage');
var system = require('system');
var page = webPage.create();

page.customHeaders = {"pragma": "akamai-x-feo-trace"};
page.settings.userAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36"

if (system.args.length === 1) {
    console.log('Try to pass some args when invoking this script!');
} else {
    page.open(system.args[1], function (status) {
    var content = page.content;
    console.log(content);
    phantom.exit();
    });
}

Now I am executing the following command to get the HTML source (generated HTML after execution of JS on the page).

phantomjs --ignore-ssl-errors=yes --ssl-protocol=any RequestURL.js #my_url_here > body.html

Now the problem is the page source does not get generated if JS errors are there on the page. Is there a way in phantoms to basically ignore any errors and get the full generated page source.

score 0 · Answer 1 · edited May 23 '17 at 12:05

If the JavaScript error is in a certain place, the execution of the complete page JavaScript can halt entirely, and if the page is a dynamic one (e.g. single page application) it will mean that the page source is effectively empty.

The only ways to go further are by either fixing the errors or preventing the errored code to execute.

For example: PhantomJS 1.x doesn't support Function.prototype.bind. Although, there exists a shim in the CasperJS source, but it doesn't work correctly, so a proper shim has to be introduced.

JavaScript errors can be fixed in many ways, but changing the source on the fly is not really an option, because PhantomJS' events don't provide access to the request content, but only to meta data. If the source is loaded through XHR, an XHR proxy or an actual proxy can help.

Sometimes page.content doesn't show the page source, but it's there. In those cases it can be retrieved through

page.evaluate(function(){
    return document.documentElement.outerHTML;
});

Some more options are here.

Phantomjs need the page source in spite of JS errors

1 Answers1