2

The SIMILE Project at MIT produced a series of tools useful for in-browser screen scraping, namely Piggy Bank, Solvent and Crowbar. These projects now appear defunct; the website has had few wiki updates since 2008. The Firefox extensions no longer work with up to date versions of Firefox (3.6). Even the mailing list archives seem to have disappeared.

Is there anything similar available, or should I consider whether I can modify the relevant parts of the SIMILE codebase to work with contemporary Firefox?

Mat
  • 82,161
  • 34
  • 89
  • 109

2 Answers2

1

What are you trying to do? If its scraping or testing a website, I found htmlunit works very well(you need jython). I had a question about this and someone on this site posted a link to screen scraping using jython Here's the link

If you want to have it drive an actual browser then take a look at Selenium I have not used it but there seems to be fairly popular.

Hope that helps!

Lostsoul
  • 25,013
  • 48
  • 144
  • 239
  • I want to process data in-browser: extract some data from the current page, do some maths on it and display it either within the same page, or more likely, in a sidebar. – Mat Mar 16 '11 at 19:24
  • I have not used it but I think you should look at Selenium. I think people use it to extract data within the browser and then run scripts against it. You can probably do that then feed the data to the side bar application your running. – Lostsoul Mar 16 '11 at 20:43
1

I have used the chickenfoot firefox extension for in-browser web scraping, which seems similar. Here is an article (from my blog) of how to use it.

Unfortunately the mailing list is dead for this project too. It seems academic funded projects have an unreliable future.

hoju
  • 28,392
  • 37
  • 134
  • 178