Automated tests for Java Swing GUIs

Question

What options are there for building automated tests for GUIs written in Java Swing?

I'd like to test some GUIs which have been written using the NetBeans Swing GUI Builder, so something that works without requiring special tampering of the code under test would be ideal.

related question: http://stackoverflow.com/questions/79891/what-is-the-best-testing-tool-for-swing-based-applications — Mike R, Apr 20 '14 at 20:42
I recommend trying ReTest (https://www.retest.de/en/), which implements an interesting new approach to functional regression testing combined with ai-based monkey testing. — roesslerj, Mar 04 '17 at 16:44

score 35 · Accepted Answer · edited Jan 29 '17 at 15:05

35

Recently I came across FEST which seemed promising, except that the developer announced in 2012 that development would not continue.

AssertJ is a fork of FEST that is working very well for me. It is actively maintained (at time of writing), supports Java 8, has assertions for a few popular libraries such as Guava and Joda Time, and is very well documented. It is also free and open.

edited Jan 29 '17 at 15:05

spongebob

8,370
15
50
83

answered Sep 24 '15 at 21:15

Jwest08

426
5
5

Can this be used to automate third party Java applications without access to the source code? If so, how? – Stevoisiak Oct 06 '17 at 17:56
@StevenVascellaro looks it cannot, anyways +1 for AssertJ Swing – emecas Dec 07 '17 at 17:20
How are you identifying the object properties. My application under test is a Java Swing UI. How do I get the properties of target fields/buttons that are to be automated? – Manmohan_singh May 01 '19 at 08:18
You can use Testmate to automate Java swing application for free https://youtu.be/BQQdJ0V23GM – Jaskaran May 19 '20 at 06:26

score 26 · Answer 2 · answered Sep 18 '08 at 11:15

26

I'm currently using FEST. It works with JUnit and will also take screenshots of failed tests.

It has default component hunting methods which look for the name of the component being tested (which need to be set manually), but you can also generate the testers for a given component by passing it the component.

answered Sep 18 '08 at 11:15

deterb

3,994
1
28
33

It moves in GitHub from [this post](http://fest.easytesting.org/archives/167). – Davide Pastore Jul 16 '13 at 20:27
5

Fest swing is not being developed any more. It does not support java 7 either. Seems like the swing part of fest is totally dead! – Apr 07 '14 at 06:30
I use FEST with Swing in Java7 just fine. See https://github.com/bitcoin-solutions/multibit-hd/tree/develop/mbhd-swing/src/test/java/org/multibit/hd/ui/fest for comprehensive test scenarios – Gary Apr 02 '15 at 15:43
1

The Fest project web site is gone (http://fest.easytesting.org). The documentation in 2015 is pretty lacking. Is there anything else like it that drives a Swing UI? – Greg Smethells Jul 01 '15 at 17:53
As someone unfamiliar with JUnit, how can I find and control a Java applet with FEST? – Stevoisiak Oct 10 '17 at 19:06
@StevenVascellaro Your best bet would be to ask a separate question, I don't do applet development. – deterb Oct 11 '17 at 22:40

score 14 · Answer 3 · edited Oct 10 '17 at 19:09

You can try to use Cucumber and Swinger for writing functional acceptance tests in plain english for Swing GUI applications. Swinger uses Netbeans' Jemmy library under the hood to drive the app.

Cucumber allows you to write tests like this:

 Scenario: Dialog manipulation
    Given the frame "SwingSet" is visible
      And the frame "SwingSet" is the container
    When I click the menu "File/About"
    Then I should see the dialog "About Swing!"
    Given the dialog "About Swing!" is the container
    When I click the button "OK"
    Then I should not see the dialog "About Swing!"

Take a look at this Swinger video demo to see it in action.

score 9 · Answer 4 · edited Jul 16 '13 at 18:39

9

We are considering jemmy to automate some of the GUI testing. Looks promising.

edited Jul 16 '13 at 18:39

talonmies

70,661
34
192
269

answered Apr 24 '10 at 12:45

Jayan

18,003
15
89
143

2

Jemmy is a very solid framework and we are using it since more than 5 years very successfully. – Mot Aug 01 '10 at 15:00

score 8 · Answer 5 · answered Sep 18 '08 at 13:20

8

I use java.awt.Robot. Is not nice, is not easy but works every time.

Pros:

You are in control
Very fast
Build your own FWK
Portable
No external dependencies

Cons:

No nice GUI to build test
You have to leave the GUI alone while you test
Build your own FWK
Difficult to change test code and create your first harness

Now if you have the budget I would go for LoadRunner. Best in class.

(Disclosure: relationship to the company that owns LR, but I worked with LR before the relationship)

answered Sep 18 '08 at 13:20

Javaxpert

2,315
1
13
4

5

You say "build your own FWK". What is a FWK? – Bryan Oakley Apr 24 '10 at 13:07
3

From my experience, I can say that AWT's `Robot` is extremely brittle. A major issue is that its low level [API](https://docs.oracle.com/javase/8/docs/api/java/awt/Robot.html) is working with absolute coordinates. Tests may break even if your system under test (SUT) hasn't changed (e.g. when executed on another OS). – beatngu13 Jun 25 '17 at 18:50

score 7 · Answer 6 · answered Sep 18 '08 at 11:39

7

We're using QF-Test and are quite satisfied.

answered Sep 18 '08 at 11:39

ahu

476
3
7

score 6 · Answer 7 · answered Sep 18 '08 at 10:25

I haven't used it personally, but SwingUnit looks quite good. You can use it with jUnit, and it isn't based on "location of components" (i.e. x and y co-ordinates).

The only thing you may have to do with the NetBeans GUI Builder is set unique names for your components.

score 4 · Answer 8 · edited Jan 06 '13 at 14:23

You can use Marathon : "Marathon Integrated Testing Environment, MarathonITE, is an affordable, easy-to-use and cross-platform Java/Swing™ GUI Test automation framework. You can use MarathonITE‘s inbuilt script recorder to create clean, readable test scripts either in Python or Ruby. Advanced features like extract-method refactoring, create-datadriven-tests and objectmap editing allows you to create maintainable, resilient test suites."

score 3 · Answer 9 · answered Jun 28 '12 at 07:09

3

Sikuli: a GUI-tester using screenshots http://sikuli.org/

answered Jun 28 '12 at 07:09

Viktor Mellgren

4,318
3
42
75

1

Note that it is http://www.sikulix.com/ nowadays (still open source) – tobi42 Dec 22 '15 at 07:06

score 3 · Answer 10 · answered Mar 04 '17 at 22:22

You could try ReTest, which is a novel tool that implements an innovative approach to functional regression testing and combines it with ai-based monkey testing. It is about to become open source as well...

Disclaimer: I am one of the founders of the company behind ReTest.

score 2 · Answer 11 · answered Jan 29 '16 at 16:45

2

For those with an adventurous mind, there is gooey https://github.com/robertoaflores/Gooey a (very basic and under-development) programmatic testing tool for swing applications.

answered Jan 29 '16 at 16:45

Roberto Flores

21
1

score 1 · Answer 12 · edited Apr 03 '19 at 07:29

1

Just did some quick scans. Squish was the most promising. Not for free though

edited Apr 03 '19 at 07:29

TheYaINN

114
10

answered Oct 21 '14 at 19:30

amutter

81
1
9

score 1 · Answer 13 · answered Mar 03 '17 at 11:16

1

You can user sikuli or Automa for testing your GUI part, these are well documented and tested tools

answered Mar 03 '17 at 11:16

Sanjit Kumar Mishra

1,153
13
32

Automated tests for Java Swing GUIs

13 Answers13

Linked