24

I'm using Python 3.6 while I have to fill in a form. Unfortunately, mechanize doesn't work on Python 3. What do you suggest as an alternative to mechanize?

Community
  • 1
  • 1
user319723
  • 451
  • 1
  • 5
  • 8

6 Answers6

15

SeleniumRC with selenium.py is an alternative (and one of the few workable options if the pages you need to scrape have an important, "structural" role for Javascript operations, esp. AJAX-y ones, since Mechanize doesn't execute the Javascript on the pages it's scraping).

Alex Martelli
  • 854,459
  • 170
  • 1,222
  • 1,395
7

For scraping and form handling you can use lxml.html (it doesn't automate fetching and cookies though).

scrapy is a library specifically for scraping.

Ian Bicking
  • 9,762
  • 6
  • 33
  • 32
5

I've been successful with Splinter, a solution built on top of Selenium - while providing more pythonic API.

K DawG
  • 13,287
  • 9
  • 35
  • 66
jmu
  • 3,619
  • 1
  • 23
  • 12
3

I've used twill for a lot of my testing needs. It works as a stand-alone language for "web browsing" or as a library from Python. It actually uses Mechanize under the hood, so I'm not sure if it will meet your needs -- are you encountering problems intrinsic to Mechanize, or would you benefit from a high level layer?

ars
  • 120,335
  • 23
  • 147
  • 134
1

scrapelib is another option : https://github.com/sunlightlabs/scrapelib

1

try zope.browser http://pypi.python.org/pypi?:action=display&name=zope.testbrowser

Yuda Prawira
  • 12,075
  • 10
  • 46
  • 54