32

Looking for a tool to UI testing of a Windows (.NET WinForms) application. The idea is that the analysts will record the tests via some UI and it will be able to be played back over and over again.

Commercial tools are fine but bonus points for free ones.

skaffman
  • 398,947
  • 96
  • 818
  • 769
Robert MacLean
  • 38,975
  • 25
  • 98
  • 152
  • 13
    Robert, with the caveat that I do not know the specifics of your project, I caution you against the record/playback strategy for test automation. Test automation is software development and needs to be treated as such. Recorded test scripts will not be resilient to change and if your app is not changing, then you don't need the automation, right? :-) – Tom E Sep 11 '09 at 15:21
  • Agree, Test automation Is software development, if you don't treat it as such and have people with the corresponding skills you won't get much out of it. – eglasius Sep 25 '09 at 17:06
  • 1
    Duplicate of http://stackoverflow.com/questions/25343/is-there-any-way-to-automate-windows-forms-testing – John Aug 16 '10 at 10:58
  • 1
    Recording is useful on the starting stages of testing to quickly build the bulk of UI map and actions. The results are then tweaked and refactored by hand to be more versatile. – ivan_pozdeev Sep 06 '13 at 12:14
  • Possible duplicate of [Is there any way to automate windows forms testing?](http://stackoverflow.com/questions/25343/is-there-any-way-to-automate-windows-forms-testing) – Dariusz Woźniak Mar 31 '17 at 00:44

12 Answers12

12

Some others:

I believe they're both free, and Quail looks really nice!

Razzie
  • 30,834
  • 11
  • 63
  • 78
10

(I was going to just comment on an answer above, but ran out of room.) We bought Ranorex, and I wouldn't do it again. Their licensing for 1.5 (what we're using) was unclear. Their written license was per user, their marketing said per machine. Still, enforcement was key-based, and not a big deal when a machine got wiped or a replacement tester came in.

For the 2.0 product, their licensing is now tied to the machine. Reimage a machine and you get to relicense it. I'm just not going to support that kind of hassle with my company's thousands of dollars, and we didn't upgrade.

For what they want for the product, a lot of the functionality could be easily written using the UI Automation Framework. Ranorex is a decent product, but I question its value for the money. We're moving a lot of our new tests to just coding to the UI Automation Framework, as we often end up modifying Ranorex-generated code anyway.

mikestew
  • 849
  • 4
  • 9
  • We're evaluating Ranorex right now, and it looks like this is much clearer now: there's a choice of node-locked licenses (anchored to a machine, but can be moved every 90 days), floating licenses (can be used on any machine, but only one user can use the license at any given time) and floating runtime licenses (only runs tests). (No, I don't work for them!) – Scottie Nov 10 '15 at 10:46
4

TestComplete.

You can definitely use capture-replay to capture and run the test script. But I would suggest that you must at least manually edit your scripts to make them

  1. more readable
  2. easier to maintain.

The good thing about TestComplete is that it is able to look into your form's properties, capture those properties so that you can refer to those properties by their name, not by just screen coordinates.

Community
  • 1
  • 1
Graviton
  • 81,782
  • 146
  • 424
  • 602
4

Like Tom E stated, do take caution while considering going down the record/playback path for test automation.

See Uncle Bob's article on Ruining your Test Automation Strategy.

The main problem is that the record/playback tools couple the tests to the GUI which makes them very fragile.

Uncle Bob's article does point out that some testing needs to occur on the GUI...but that he recommends stubbing out the business rule code.

Sorry I can't provide you with a specific UI test automation tool...but hopefully this caveat will help you make the best decision on how to employ the tool that you eventually use.

Community
  • 1
  • 1
mezoid
  • 28,090
  • 37
  • 107
  • 148
3

So far I have found:

Comments please if you have used any of these.

Robert MacLean
  • 38,975
  • 25
  • 98
  • 152
  • 1
    White is a wrapper around the UI Automation Framework. It adds a neat hook that waits for the process's input thread to go idle before proceeding to the next command, which ultimately means it runs as the speed of your machine and doesn't need Thread.Sleep calls. – bryanbcook Sep 30 '09 at 05:04
2

There are a bunch of similar questions on SO:

  • automate-interaction-with-a-gui-interface (Edit: no longer available)
  • automated-testing-of-windows-forms (Edit: no longer available)
  • automated-testing-of-gui

In my experience, there are a lot of good open source tools for the web, but not so much selection for open source thick client test automation tools. If you want good support with robust functionality, especially recording, you will need to look at the commercial tools (QTP, RFT, TestPartner, etc...)

Community
  • 1
  • 1
Tom E
  • 2,519
  • 2
  • 17
  • 22
2

At my company, we decided to go with http://www.sikuli.org/. We felt it was the perfect mix of cost (free), ease-of-use, functionality and extend-ability.

Sikuli is Python based (Jython, really) which is great and its open-source. There's a tiny bit of coding required, but it can be as easy as just calling functions. Their IDE makes it really easy to get started. It is not a record and playback tool. It functions based on computer vision algorithms - you give it screenshots of what to look for and it finds it on the screen and then performs the requested action (click, type, etc) on what it found. This is true independent testing since Sikuli knows nothing about the software its testing. It does not know about underlying APIs - it just does what a real human would do.

We have integrated Sikuli with Robot Framework, http://robotframework.org/, and have created lots of custom python code to build a robust testing platform. This may not be as easy as licensing a tool from a vendor but the time and talent investment in these two open source tools has been well worth it.

cbautista
  • 330
  • 3
  • 6
1

You should have a look at http://opensourcetesting.org/functional.php A lot of tools are listed here and you should find something that meet your needs.

luc
  • 41,928
  • 25
  • 127
  • 172
1

Visual Studio Team Test 2010 is coming with a tool for recording and playing back UI tests. You'll find some pointers at:

Introduction to Record and Playback Engine in VSTT 2010

Alfred Myers
  • 6,384
  • 1
  • 40
  • 68
0

Checkout Ranorex, commercial, quite expensive but powerful (not affiliated).

zzandy
  • 2,263
  • 1
  • 23
  • 43
0

We were using Mercury TestDirector a few years ago and quite happy with it. (All the caveats as mentioned by others apply.)

Mercury was aquired by HP and the tools have been rebranded as HP QuickTest. Not sure how much has changed, but certainly worth a look.

I tried to include a link to the HP website, but the URL doesn't look too "stable". Not exactly a confidence builder ....

Thorsten
  • 12,921
  • 17
  • 60
  • 79
  • The actual GUI test was called "Winrunner" and was very splendid last time I used it some years ago. I agree with the caveat about HP. "Your software has entered an endless swamp, where everything is rebranded and renamed , but nothing ever really changes -- an annoying hardware salesman trys to sell you an itanium based workstation" – James Anderson Oct 01 '09 at 02:15
0

My proposal is QA Agent(http://qaagent.com/). It is free web based IDE for development of web automated tests. Basically you are using jQuery to develop your tests. You can run tests in your browser. It looks like a new projects, but I really like the idea to develop tests in the browser. You know how long it takes to set up your testing environment. With QA Agent it takes 10 seconds.

Angel Tsvetkov
  • 401
  • 2
  • 9