0

Possible Duplicate:
How can I use Perl to grab text from a web page that is dynamically generated with JavaScript?

I am interested in learning Perl and just started to write a web crawler in Perl. On the page that I crawled, there is a Javascript for giving vote to the page. It shows the vote of the current page, only if I click the voting stars. So during the crawling, I need to run the Javascript and learn the current voting of the page.

Do you have any suggestions or examples?

Thanks..

Community
  • 1
  • 1
perloc
  • 101
  • 1
  • 10
  • 1
    Topic was discussed already very often. Search for perl javascript, relevant: And these are just from the first result page. Mech FAQ: [Which modules work like Mechanize and have JavaScript support?](http://p3rl.org/WWW::Mechanize::FAQ#Which-modules-work-like-Mechanize-and-have-JavaScript-support-) – daxim Apr 11 '12 at 09:21
  • 1
    Submitting fake votes in order to get the data strikes me as being an exceptionally bad idea. – Quentin Apr 11 '12 at 11:00

2 Answers2

3

You would need to use module WWW::Scripter with WWW::Scripter::Plugin::JavaScript.

Synopsis:

#!/usr/bin/perl

use strict;
use warnings;
use WWW::Scripter;

$w = new WWW::Scripter;
$w->use_plugin('JavaScript');  # packaged separately
$w->get('http://some.site.com/that/relies/on/javascript');
$w->eval(' alert("Hello from JavaScript") ');
$w->document->getElementsByTagName('div')->[0]->....
$w->content; # returns the HTML content, possibly modified by scripts
Ωmega
  • 42,614
  • 34
  • 134
  • 203
2

Using perl to drive an actual browser is one option (e.g. WWW::Mechanize::Firefox, WWW::Selenium, etc.). To actually run javascript in perl, try WWW::Scripter with WWW::Scripter::Plugin::JavaScript.

ysth
  • 96,171
  • 6
  • 121
  • 214