What is the best testing tool for Swing-based applications?

Question

While we try to set up as many unit tests as time allows for our applications, I always find the amount of UI-level tests lacking. There are many options out there, but I'm not sure what would be a good place to start.

What is your preferred unit testing tool for testing Swing applications? Why do you like it?

related: http://stackoverflow.com/questions/91179/automated-tests-for-java-swing-guis — Mike R, Apr 20 '14 at 20:41
I recommend trying ReTest (https://www.retest.de/en/) which implements an interesting new approach to functional regression testing combined with monkey testing. — roesslerj, Mar 04 '17 at 16:42

score 11 · Accepted Answer · answered Sep 17 '08 at 05:15

11

On our side, we use to test SWING GUI with FEST. This is an adapter on the classical swing robot, but it ease dramatically its use.

Combined with TestNG, We found it an easy way to simulate "human" actions trough the GUI.

answered Sep 17 '08 at 05:15

gizmo

11,819
6
44
61

Hi @gizmo I am also working on some tests with fest using testng. Is that possible to launch an '.jnpl' java application and then simulate 'human' actions on that dialog or window? – Zzz... Jun 23 '15 at 21:56
As already referred to in Mike's comment on the question, the current [answer](http://stackoverflow.com/a/32771043/5127499) to this might be [AssertJ Swing](https://joel-costigliola.github.io/assertj/assertj-swing.html), which is a fork of FEST Swing that is still being maintained. – Carsten Jul 22 '16 at 09:48
Another possibility is ReTest, which also bases on FEST and implements a promising new approach to regression testing: https://www.retest.de/en/ – roesslerj Mar 04 '17 at 16:39
@Carsten As a complete novice, how would we start using FEST? Can it be imported into Eclipse? – Stevoisiak Oct 10 '17 at 18:27
@StevenVascellaro so far I‘ve only used AssertJ Swing for a couple tests in my pet project SciToS. The tests I wrote there pretty much look like unit tests. Maybe ask this as a separate question and share that one with me. Happy to elaborate a bit more there then. Wouldn’t know where to start in a comment here. – Carsten Oct 10 '17 at 20:26
@Carsten I think "How do I use the FEST library" would be closed as too broad for Stack Overflow. – Stevoisiak Oct 10 '17 at 20:27
1

@StevenVascellaro fair enough, I‘ll see whether I can write a question+answer myself then and share it with you afterwards. – Carsten Oct 10 '17 at 20:29
@Carsten I have been struggling to make headway on automating Swing applets, so your help is greatly appreciated. – Stevoisiak Oct 10 '17 at 20:30
1

@StevenVascellaro I wrote a question+answer now: https://stackoverflow.com/q/46676726/5127499 I hope this points you in the right direction. – Carsten Oct 10 '17 at 22:02
@gizmo Will FEST work for JAVA 8 Version Developed Swing Application? – JAVA Jan 21 '19 at 06:51

score 6 · Answer 2 · answered Nov 04 '09 at 02:10

If your target application has custom components, I would definitely recommend Marathon to automate your tests.

I was given the task of automating an application with several extremely complicated custom components, written in-house from the ground up. I went through a review process that lasted two months, in which I made the decision on which test tool to use, from a list of close to 30 test tools that were available, both commercial and FOSS.

It was the only test tool that was able to successfully automate our particular custom components; where IBM's Rational Functional Tester, Microfocus' TestPartner, QF-Test, Abbot & FEST failed.

I have since been able to successfully integrate the tests with Cruise Control such that they run upon completing each build of the application.

A word of warning though:
1) it is rather rough around the edges in the way it handles JTables. I got around this by writing my own proxy class for them.
2) Does not support record/replay of drag-and-drop actions yet.

Very interesting informations. Any feedback on http://frankenstein.openqa.org/? — Pascal Thivent, Nov 04 '09 at 02:20
Hi Pascal, unfortunately Frankenstein wasn't one of those that I did try. I looked at the presentation on the link, and I can say that it does look rather similar to Marathon in terms of proxy classes and OGNL-style assertions. Differences: Marathon doesn't appear to have inbuilt threading support or i18n support. Looks interesting. — bguiz, Nov 04 '09 at 02:42

score 4 · Answer 3 · answered Sep 23 '08 at 16:56

4

Consider Marathon (http://www.marathontesting.com/Home.html)--tests are written in Jython, so it's easy to write any sort of predicates based on object state.

answered Sep 23 '08 at 16:56

score 2 · Answer 4 · answered Jun 16 '09 at 11:51

You can try to use Cucumber and Swinger for writing functional acceptance tests in plain english for Swing GUI applications. Swinger uses Netbeans' Jemmy library under the hood to drive the app.

Cucumber allows you to write tests like this:

 Scenario: Dialog manipulation
    Given the frame "SwingSet" is visible
      And the frame "SwingSet" is the container
    When I click the menu "File/About"
    Then I should see the dialog "About Swing!"
    Given the dialog "About Swing!" is the container
    When I click the button "OK"
    Then I should not see the dialog "About Swing!"

Take a look at this Swinger video demo to see it in action.

score 2 · Answer 5 · answered Oct 10 '12 at 14:39

Not an answer, but a refining.

Record-and-playback is the wrong thing to want. Teams need the ability to write tests before the code has been written. Otherwise, the coders finish their work and wait around while the testers scramble to record tests (interrupted by fixes when they spot issues).

In a BDD/TDD/ATDD kind of setup, you really need some kind of tool that allows you to script tests for code that hasn't been written yet, specifying UI element names and the like.

Are there tools that work for non-waterfall testing?

score 2 · Answer 6 · answered Jun 06 '11 at 01:03

I can highly recommend QFTest. I have used it for my commercial product and it works very well with almost zero code (my app requires the use java client APIs for some things). It handles identification of swing components well, and is pretty tolerant of updates to your GUI - (resizing,repositioning and adding components does not break existing tests). I have done major updates to functionality and have my tests still work.

Its expensive, but I think it will pay itself off in a couple of months.

Before QFTest I tried:

1) Automatedqa - good tool, but windows centric and does not understand Swing. Similar to Quick test Pro.

2)UISpec4J - After devoting a solid 50 hour week to this, I had issues with fragility and the arcane java code it produced. Using it was just too arduous - trying to debug/update hundreds of lines of java performing a sequence of a dozen GUI operations just did not work for my brain. I ended up avoiding writing tests because it much more complicated than actually writing the app itself!

score 2 · Answer 7 · answered Sep 17 '08 at 19:59

2

I had the chance to play around with QF-TEST once. It is commercial, but offers a lot of functionality. Maybe you have a look at it: http://www.qftest.de/en/index.html

answered Sep 17 '08 at 19:59

Roland Schneider

3,615
3
32
43

score 1 · Answer 8 · answered Sep 26 '08 at 22:37

1

I like Jemmy, the library written to test Netbeans.

answered Sep 26 '08 at 22:37

Tom

55,743
3
27
35

score 1 · Answer 9 · answered Sep 17 '08 at 05:06

1

try pounder : http://pounder.sourceforge.net/

answered Sep 17 '08 at 05:06

zurk

249
1
2
5

What is the best testing tool for Swing-based applications?

9 Answers9

Linked