1

Because Selenium can traverse javascript websites (which Mechanize cannot), and Mechanize can make post requests (which Selenium cannot), in some cases it would be powerful to use the two in conjunction.

The answer by +Zarkonnen to this question suggests that one would use Selenium initially, then Mechanize would step in to make the post request and than pass that back to Selenium.

How would one integrate Mechanize post method into Selenium?

I am using the Ruby versions of these libraries, but any information would be useful.

enter image description here

EDIT Here's a Venn Diagram to hopefully clarify the functionality I am seeking.

"Javascript website" in this case simply means a website whose functions in question will not work without javascript enabled. Meaning, say I needed to traverse a website to get to a form on that website. Along the way I ran into buttons which didn't work without javascript enabled. Then, in order for the form to work the way I wanted, I had to do a custom post. In this case scenario, neither Selenium WebDriver nor Mechanize can handle it by themselves - they need help from each other.

How would you accomplish this? Would you use Selenium and then have Mechanize step into to help when you had to do the post? Would you use some other method to make a post within Selenium? Would you use the Capybara gem? I get there are limitations with WebDrivers making Posts, but I know there must be a workaround.

Community
  • 1
  • 1
singularity
  • 573
  • 4
  • 15
  • Generally speaking, people use Capybara instead of Mechanize to do this kind of stuff. You can configure Capybara to use the selenium driver (or webkit or poltergeist/phantomjs) and achieve the same results. It's much more powerful than mechanize with a fairly easy DSL syntax. Maybe try that. – Josh Deeden Feb 26 '16 at 22:41
  • @Josh - no people do not generally use capybara, selenium (watir) is actually a more popular ruby library. – pguardiario Feb 27 '16 at 09:20
  • It's hard to tell what you're asking, are you wondering how to pass cookies back and forth, or maybe how to turn a Mechanize::Page body into Selenium browser html? – pguardiario Feb 27 '16 at 09:23
  • @pguardiario you are in fact, wrong.. at the time of this comment, capybara has 6,815 stars on Github, where as watir has 377. On https://www.ruby-toolbox.com/categories/browser_testing, Capybara is #1 and watir is hardly in the top 10. Your statement may have been true like, 10 years ago. – Josh Deeden Feb 27 '16 at 15:17
  • @pguardiario I added some clarification to the original post. Your comment might hold the answer to what I am seeking. If Se cannot make a post, but you can hand off an Se session to Mechanize, and then hand it back, then maybe that's the key to filling in the utility gaps? (See the Venn diagram in the edit in the post). – singularity Feb 27 '16 at 16:34
  • I think there's a fundamental confusion here between interactive and non-interactive use, as demonstrated by the linked-to question. What it boils down to is: in interactive mode (i.e. simulating a user), why do you think you need to do a custom POST at all (rather than interactive `form`s that use POST, which Sel handles perfectly well)? See my answer for some scenarios. But this is not a limitation at all, simply two different modes. Combining the two is generally a bad idea. – Andrew Regan Feb 27 '16 at 18:10
  • Also, the capybara / watir discussion is a bit misleading, as both (can) use WebDriver to drive browsers (i.e. interactive mode). Any Ruby framework can potentially execute HTTP code (i.e. non-interactive mode), though they will have their own specialities. But just changing to/from Mechanize, capybara, watir won't necessarily change anything. – Andrew Regan Feb 27 '16 at 18:17
  • @AndrewRegan The reason I think I need to do a custom post is because there's an ASP form I'm working with which the Selenium-WebDriver in Capybara is having a hard time posting to. It was a while back, but for whatever reason, I ended up landing on Mechanize as a solution because for the life of me I couldn't find a way to post via user-simulated interaction. I had to dive in and replicate the post - as ugly as it sounds, Mechanize does give a lot more control in this area. – singularity Feb 28 '16 at 14:28
  • 1
    Combining interactive and non-i is obviously very tricky due to having to pass all the same cookies in order to recreate the user's state in non-i mode. I wouldn't do that unless I was desperate. I'd probably prefer to do everything non-interactively. Definitely raise another question here to deal with any browser-posting issues you have rather than go down that route IMO. – Andrew Regan Feb 28 '16 at 14:37
  • @AndrewRegan - "very tricky due to having to pass all the same cookies" - So it can be done, it's probably just very complicated and messy. Thanks for your comment. – singularity Feb 28 '16 at 14:55

1 Answers1

1

The question is a bit vague, but both Selenium (WebDriver) and a good non-interactive HTTP library (like Mechanize) are crucial elements in a tester's armoury.

In general I say that if you need to simulate a human being in an interactive scenario, then you can't beat WebDriver. However, the web is built upon HTTP, everything Selenium does is HTTP, and so the less interactive your scenario, the less you need to simulate a real user, and the more performance matters, the more you should look to Mechanize, and possibly even lower-level HTTP libraries.

Because of that, although the two technologies are complementary in a sense, I can't think of all that many good reasons to use them in conjunction. But perhaps the following:

  • WebDriver manages a user on a web site, Mechanize is used to query REST endpoints to dump metrics, clear caches, run usage reports, kick off simultaneous requests to simulate concurrency.
  • Mechanize is used to seed/prepare test data prior to a WebDriver run.

Those are both examples where WebDriver could be used for everything, but where it would be vastly easier and more efficient to use a non-interactive tool.

Andrew Regan
  • 5,087
  • 6
  • 37
  • 73