Java(or any lang) library for offline rendering of web pages?

Question

I am developing a java application. I have scenario to take screen shot of the URL that comes in to the server.

Is there any java(or any lang) browser library to load webpages and get some screenshots of the loaded page. It would be nice if the lib allows DOM traversal.

---

Update:

java(or any lang): Any other language is not a problem but the library should co-operate with java.

I have tried to setup Qt Jambi and spent a lot of time on this but the result is nothing. If you provide any concrete material to setup Jambi, it would be appreciative.

I also gave a try to spynner.py. My native language is Java and i thought i could use spynner.py with Jython. But, PyQt cannot be used with Jython. So, i am not expecting any answers related to Python.

---

Basically, I need a library to do:

• Take Screen shot.

• Some DOM traversing.

• Some Javascript Execution.

• and to get the result of the Executed JS code.

Thanks.

---

I appreciate all the responses. I ended up with phantomjs. It fits well for my needs. Its a command line tool.

Answer 1

Selenium/Webdriver provides all this functionality.

Webdriver provides a simple api allowing you to "drive" a browser instance. Many browsers are supported.

See here for a simple example:

http://seleniumhq.org/docs/03_webdriver.html#getting-started-with-selenium-webdriver

Traversal of the dom using the "By" locators:

Good examples here: http://www.qaautomation.net/?p=388

driver.findElement(By.name("q"));

Execution of Javascript:

http://code.google.com/p/selenium/wiki/FrequentlyAskedQuestions#Q:_How_do_I_execute_Javascript_directly?

WebDriver driver; // Assigned elsewhere
JavascriptExecutor js = (JavascriptExecutor) driver;
js.executeScript("return document.title");

Screenshot capture:

http://seleniumhq.org/docs/04_webdriver_advanced.html#taking-a-screenshot

 File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);

Answer 2

In java, you should read the following stackoverflow posts :

Programmatic web browser Java library

Take a screenshot of a webpage with JavaScript?

Embed a web browser within a java application

---

Because you say "or any lang" :

In Python, you have Spynner :

Spynner is a stateful programmatic web browser module for Python with Javascript/AJAX support based upon the QtWebKit framework.

According to the documentation, here's a small snippet :

import spynner

browser = spynner.Browser()
browser.load("http://www.wordreference.com")
browser.runjs("console.log('I can run Javascript!')")
browser.runjs("_jQuery('div').css('border', 'solid red')") # and jQuery!
browser.select("#esen")
browser.fill("input[name=enit]", "hola")
browser.click("input[name=b]")
browser.wait_page_load()
print browser.url, len(browser.html)
browser.close()

Answer 3

This site does the screenshot job:

Tutorial: http://www.paulhammond.org/webkit2png/

The program: http://www.paulhammond.org/2009/03/webkit2png-0.5/webkit2png-0.5.txt

Could it be any easier ? :)

There are some other tools mentioned at that page:

" If you use a mac, but don't like the command line then you may want to try Paparazzi or Little Snapper. If you use linux you may be more interested in khtml2png, Matt Biddulph's Mozilla screenshot script or Roland Tapken's QT Webkit script. "

You could use Rhino, Gecko for the javascript execution.

For dom traversal there are many options, but if you are using Rhino you could use jQuery to make it even easier!

Hope that works out for you!

Answer 4

If you need a screenshot, I guess the quality of rendering is important for you. We had a similar scenario. What we ended up doing is to run firefox on headless mode, actually browse the webpage and get a screen shot in memory. It is not trivial, but I can give you more details if you wanted to go for it.