Android Unit Test Not Reporting Failing with fail()

Question

I've written a unit test that simply extends TestCase and I have the following:

public class MetricParserTests extends TestCase {

    @Override
    protected void setUp() throws Exception {
        super.setUp();
    }

    @Override
    protected void tearDown() throws Exception {
        super.tearDown();
    }

    public void testFailure() {
        fail("This needs to fail");
    }
}

When I run my tests using ant test or adb shell am instrument I get the following results:

... [exec] OK (1 tests) ...

I'd expect to see a failure on the command line.

I am having the same problem. I can see the AssertionFailedError is thrown when I run line by line, but eclipse, ant and logcat all report the test has passed? — Chris Noldus, Jan 13 '14 at 01:58
Why duplicate it? It is very likely we have the exact same problem. — Chris Noldus, Jan 13 '14 at 02:22

sergej shafarenka · Accepted Answer · 2014-01-19T18:55:30.810

1

I believe I know what the issue is. I was able to reproduce the issue and solve it. The command you use does not rebuild and re-install your test project onto a device. When you call ant test it will just execute the tests which are already installed on that device.

What you need to call is the three commands in your test project's directory:

ant debug
ant installd
ant test

Then all tests will be rebuild and re-installed and latest tests will be executed. If you don't call debug and installd, the changes you did to the tests do not get applied.

edited Jan 19 '14 at 18:55

answered Jan 19 '14 at 14:51

sergej shafarenka

20,071
7
67
86

This answer does not help me - I see the question has been accepted by Gregg, however reading his question indicates that this should not be correct, as his code only had one test and it was being executed. My code only has 5 tests, and they should all fail, the fact that all 5 come up with a pass, clearly shows that the latest code is on the device (Not to mention I did try removing and installing at least once). As per Dagon's suggestion, I will re-ask this question with the details of my particular situation. – Chris Noldus Jan 20 '14 at 22:26
1

@ChrisNoldus feel free to open a new question describing your issue in details and community will try to help you. – sergej shafarenka Jan 21 '14 at 07:01

score 0 · Answer 2 · edited May 23 '17 at 11:57

I haven't had recent experience in Android testing, but here is what I have found...

You can use normal JUnit tests if your code is totally decoupled from Android (see here for an example). This would run on your JVM using the JUnit runner.

However, if you are trying to run these tests on an Android device (either via ant, or the command line tools) then you need to create a full android test project (See here). To test "on device" your test cases need to extend one of the Android test classes like ActivityInstrumentationTestCase2<T> and are run using the InstrumentationTestRunner in the Dalvik VM on the Android device. Using an IDE or the command-line tools to create a test project should create a sample test for you to work from.

This blog post linked from the comments of the post above is a good source of information, as is the Android Testing Fundamentals doc.

I can not speak for Gregg, but this answer does not help me. I am using an Android Test Project, and i am extending an android test case (In my scenario, it's ApplicationTestCase). I do need to test in the Android environment so JUnit on it's own would not be helpful. — Chris Noldus, Jan 14 '14 at 02:33

score -1 · Answer 3 · answered Dec 17 '13 at 20:03

-1

The method testFailure() does not have a @Test annotation. Is that correct?

answered Dec 17 '13 at 20:03

Vlad.Bachurin

1,340
1
14
22

1

No, Android uses JUnit3, so no annotations. – Gregg Dec 17 '13 at 20:27

Android Unit Test Not Reporting Failing with fail()

3 Answers3