I am trying to figure out, how to re-run failed tests with usage of Espresso. I think it's a bit more complicated from common JUnit test case as you need to restore status in your app from before test start.
My approach is to create my own ActivityTestRule so I just copied whole code from this class and named it MyActivityTestRule.
In case of instrumentation tests rule will also need information of how we want to start our activity. I prefer to launch it myself rather than have environment to do it for me. So for example:
@Rule
public MyActivityTestRule<ActivityToStartWith> activityRule = new MyActivityTestRule<>(
ActivityToStartWith.class, true, false
);
So I also launch my activity in @Before annotation method:
@Before
public void setUp() throws Exception {
activityRule.launchActivity(new Intent());
}
And make clean up in @After annotation method:
@After
public void tearDown() throws Exception {
cleanUpDataBaseAfterTest();
returnToStartingActivity(activityRule);
}
Those methods - setUp(), tearDown() are essential to be called before/after each test run - to ensure the app state during the test start is correct.
Inside MyActivityTestRule I did few modifications so far. First thing is change of apply method from:
@Override
public Statement apply(final Statement base, Description description) {
return new ActivityStatement(super.apply(base, description));
}
It's a but unknown thing for me, as ActivityStatement placed in ActivityTestRule has super.apply method so it also wraps test statement in UiThreadStatement:
public class UiThreadStatement extends Statement {
private final Statement mBase;
private final boolean mRunOnUiThread;
public UiThreadStatement(Statement base, boolean runOnUiThread) {
mBase = base;
mRunOnUiThread = runOnUiThread;
}
@Override
public void evaluate() throws Throwable {
if (mRunOnUiThread) {
final AtomicReference<Throwable> exceptionRef = new AtomicReference<>();
getInstrumentation().runOnMainSync(new Runnable() {
public void run() {
try {
mBase.evaluate();
} catch (Throwable throwable) {
exceptionRef.set(throwable);
}
}
});
Throwable throwable = exceptionRef.get();
if (throwable != null) {
throw throwable;
}
} else {
mBase.evaluate();
}
}
}
No mather what I do with my tests I can never create case mRunOnUiThread boolean to be true. It will be true if within my test cases, tests with annotation @UiThreadTest will be present - or that's what I understood from code. It never happens though, I don't use anything like that so I decided to ignore this UiThreadStatement and change MyActivityTestRule to:
@Override
public Statement apply(final Statement base, Description description) {
return new ActivityStatement(base);
}
And my test cases run without any problem. Thanks to that all I have left - that wraps around mBase.evaluate() is:
private class ActivityStatement extends Statement {
private final Statement mBase;
public ActivityStatement(Statement base) {
mBase = base;
}
@Override
public void evaluate() throws Throwable {
try {
if (mLaunchActivity) {
mActivity = launchActivity(getActivityIntent());
}
mBase.evaluate();
} finally {
finishActivity();
afterActivityFinished();
}
}
}
In general launchActivity will be called only if I set in the 3rd parameter of ActivityTestRule constructor value true. But I launch tests by myself in setUp() so it never happens.
From what I understood mBase.evaluate() runs my code inside @Test annotation method. It also stops the test case during throwable being thrown. That means I can catch it and restart it - like proposed there: How to Re-run failed JUnit tests immediately?
And okay I did something like that:
public class ActivityRetryStatement extends Statement {
private final Statement mBase;
private final int MAX_RUNS = 2;
public ActivityRetryStatement(Statement base) {
mBase = base;
}
@Override
public void evaluate() throws Throwable {
Throwable throwable = null;
for (int i = 0; i < MAX_RUNS; i++) {
try {
mBase.evaluate();
// if you reach this lane that means evaluate passed
// and you don't need to do the next run
break;
} catch (Throwable t) {
// save first throwable if occurred
if (throwable == null) {
throwable = t;
}
// my try that didn't work
launchActivity(testInitialIntent);
// I've returned test to starting screen but
// mBase.envaluate() didn't make it run again
// it would be cool now to:
// - invoke @After
// - finish current activity
// - invoke @Before again and start activity
// - mBase.evaluate() should restart @Test on activity started again by @Before
}
}
finishActivity();
afterActivityFinished();
// if 1st try fail but 2nd passes inform me still that there was error
if (throwable != null) {
throw throwable;
}
}
}
So those comments in catch block are parts I don't know how to do. I tried to perform launchActivity on intent I used in setUp() to run the test for the 1st time. But mBase.evaluate() didn't make it react (test case didn't go again) - nothing happened + it wouldn't really save me there I think. I lacked some initiations I do in @SetUp, it wasn't called again. I would really like to find a way how to properly restart whole test lifecycle @Before @Test @After over again. Maybe some call on Instrumentation or TestRunner from code.
Any thoughts about how it could be done?
