AndroidJUnit4.class + org.junit.Assume.assumeTrue = AssumptionViolatedException

Question

I've managed to get my Android project transitioned over to JUnit4, and of course the main reason I wanted to do it isn't working. Would love any help if anyone's got ideas here.

The problem I'm trying to solve is that I want to automatically skip certain tests if the build is not pointed at the staging server. I've got this set up with a BUILD_TYPE which is using gradle to inject the base URL.

I set up an assumeThat clause in my setup which correctly identifies when the build is not staging, but instead of halting and ignoring the rest of the test, it throws an exception and fails.

Here's my base class for my live API tests - I've annotated descending from this with @RunWith(AndroidJUnit4.class), so in theory this should always be run with the JUnit4 runner:

package com.[my package].nonuitests.liveservertests;

import android.support.test.runner.AndroidJUnit4;
import com.[my package].nonuitests.BaseAndroidTest;
import org.junit.Test;
import org.junit.runner.RunWith;

/**
 * Tests against the live API. All tests descending from this class will
 * be ignored if the BUILD_TYPE is not staging.
 */
@RunWith(AndroidJUnit4.class)
public class BaseLiveServerTests extends BaseAndroidTest {

    private static final String STAGE = "staging";

    /******************
     * SETUP/TEARDOWN *
     ******************/

    @Override
    public void setUp() throws Exception {
        super.setUp();

        //TODO: Y U NO WORK?!
        //This should cause the rest of the test to be skipped if it fails,
        //but is instead throwing an AssumptionViolatedException. 
        assumeTrue(STAGE.equals(BuildConfig.BUILD_TYPE));
    }
}

So, my questions:

Is there a better way to do this? In theory this could be done with flavors, but I was trying that earlier and it made everything else way more complicated.
My research indicates there's some kind of thing that Google's not implementing in their runner that's causing this to bomb out, but I'm having a hell of a time figuring out what I can/should do to fix this. Any suggestions of things I should subclass to get this to work as expected?

Any other thoughts would be most appreciated. Thanks!

Edit (1/26/15 11:40am CST): Per Grzesuav's suggestion, I took a stab at implementing this as an @Rule, but at the moment it's still not working. This seems a promising path, but it ain't working at the moment.

Edit 2 (1/26/15 12:15pm CST): OK, now it's working.

Can you provide the code for `BaseAndroidTest`? If it indirectly extends `junit.framework.TestCase`, then that might be your problem. JUnit4-style tests shouldn't extend `TestCase`. Also, generally test-level setup in JUnit4 happens in a public method annotated with `@Before` — NamshubWriter, Jan 26 '15 at 16:55
It does not extend anything. And yes, I caught that error, and fixing it still doesn't do anything. :( — DesignatedNerd, Jan 26 '15 at 17:37
I have open an issue on the android-test-kit page regarding this: https://code.google.com/p/android-test-kit/issues/detail?id=126 — Bill, Jan 27 '15 at 00:36
@Bill I saw your bug report was marked as fixed in espresso 2.1, could you please confirm that ? I am having same issue with espresso 2.1 - see my comment below google's responses. — kiruwka, Apr 25 '15 at 21:38
Oof. I'm going to be updating some tests to 2.1 later this week, so I'll take a look. Only thought before trying to update: Have you verified you've got the updated runner? — DesignatedNerd, Apr 27 '15 at 18:40

score 2 · Accepted Answer · answered Jan 26 '15 at 08:39

2

https://github.com/junit-team/junit/wiki/Assumptions-with-assume

ad 2) Custom runners could differently treat assume statement. To fix it you should write own version of Android runner and implement a way of dealing with assumes as native JUnit runner does or make a bug for android test runner.

ad 1) Suggested by me : try use JUnit Rules :

http://www.codeaffine.com/2013/11/18/a-junit-rule-to-conditionally-ignore-tests/ http://cwd.dhemery.com/2010/12/junit-rules/

answered Jan 26 '15 at 08:39

Grzesuav

459
2
10

Alas, writing my own version of the Android test runner is probably outside of the scope of my skills/time. I took a look at the JUnit rules stuff and it's mostly relying on `MethodRule`, which is deprecated. I tried both that and extending TestRule (what I did is here https://gist.github.com/designatednerd/153e4545af912aeed1ff) - in both cases, in debug mode, breakpoints I set throughout the ConditionalIgnoreRule class were never hit. – DesignatedNerd Jan 26 '15 at 17:38

score 2 · Answer 2 · edited May 23 '17 at 10:29

2

OK, finally got it working with @Rules per Grzesuav's suggestion, although with significant changes since MethodRule has been deprecated. Here's a gist of what it turned out to be - I'll try to keep that updated as I refine it.

Some important notes:

You have to instantiate your @Rule in your test class, or you'll never actually hit any of your checks.
As of right now, this will not mark the test as ignored on Android, it'll just pass it without actually testing anything.

edited May 23 '17 at 10:29

Community

1
1

answered Jan 26 '15 at 18:16

DesignatedNerd

2,514
1
23
46

JUnit undeprecated `MethodRule`. You can use it for any version of JUnit >= 4.7 without any concerns. – NamshubWriter Jan 27 '15 at 01:05
@DesignatedNerd By "pass it" you mean skip the test case or mark it as pass? – Marcelo Dec 15 '15 at 21:12
1

@marcelo: Mark it as passed. – DesignatedNerd Dec 17 '15 at 16:50

score 0 · Answer 3 · answered Mar 07 '20 at 04:59

In Junit 4.12 it cannot handle tearDown

If you have tearDown you have to add an if statement with your condition rather than assumeTrue. I think the owners of Junit say it isn't supposed to work with @After

@After
    override fun tearDown() {
        if (junit == worksAgain()) {

AndroidJUnit4.class + org.junit.Assume.assumeTrue = AssumptionViolatedException

3 Answers3

Linked