Execute all javascript in a webpage before parsing

Question

I want to download the html of a web page containing some javascript. If I use a library like jsoup, I obtain the html without the elements generated by the javascript.

How can I take the html obtained after the javascript execution?

edit: How can I use the script in the answer in Java program?

If you want to *execute* (not just read) the JavaScript on a web site, you need to run [a headless Web browser](https://gist.github.com/evandrix/3694955) instead of using an HTML parser. — Jacob Budin, Jan 03 '15 at 16:10

score 3 · Accepted Answer · edited May 23 '17 at 12:30

3

You can use PhantomJS with the following script:

var page = require('webpage').create();
page.open('http://stackoverflow.com',function(status){
  if(status !== 'success'){
    console.log('Open failed');
  }else{
   console.log(page.evaluate(function(){
                               return document.documentElement.outerHTML;
                             }));
  }
  phantom.exit();
});

If you want execute this script from Java read the following :

Running Phantomjs from javascript, JSP or Java

edited May 23 '17 at 12:30

Community

1
1

answered Jan 03 '15 at 19:38

Ortomala Lokni

56,620
24
188
240

thanks for the answer. But I have always a problem: If I use this script with the code shown in the link, my program go in loop and I have no answer. What can i do to resolve this problem?? thanks – Frank Cunningham Jan 04 '15 at 10:02
The best is to ask a new question with full details. – Ortomala Lokni Jan 04 '15 at 10:04

Execute all javascript in a webpage before parsing

1 Answers1