13

I've discovered a problem recently with my app's setup in Firebase. We have the pre-launch report configured, which means that the test lab devices are contributing a large number of anonymous sessions to our analytics.

Is there any way to detect that a device is running these tests? For example, will isUserAMonkey or isRunningInTestHarness return true for tests in the test lab? This would allow me to adjust the configuration of my analytics/etc if so.

tmtrademark
  • 944
  • 6
  • 21

2 Answers2

24

This is actually mentioned in the docs.

You can check for the existence of a system property called "firebase.test.lab":

@Nullable String testLabSetting =
  Settings.System.getString(context.getContentResolver(), "firebase.test.lab");
if ("true".equals(testLabSetting)) {
  // Do something when running in Test Lab
}
guy.gc
  • 3,359
  • 2
  • 24
  • 39
Doug Stevenson
  • 297,357
  • 32
  • 422
  • 441
  • 2
    One follow-up question here. Using this code seems to have detected most of the devices running automated tests, but not all. The Play pre-launch report indicates that tests were run on a Nexus 5/7/9 and a Galaxy phone. Those devices have stopped appearing in our logs - but one rogue Nexus 5X continues to appear with each build. Is it possible that there is some other configuration variable that needs to be checked in addition to the "firebase.test.lab" property mentioned here? – tmtrademark May 10 '17 at 03:31
  • Not to my knowledge. Please feel free to join the #test-lab channel on the Firebase Slack to ask more about this. If there is a problem, we'll look into it. https://firebase.community/ – Doug Stevenson May 10 '17 at 06:49
  • Note that if you're converting this answer to Kotlin, testLabSetting can be null so you'll need to declare `testLabSetting` as `String?` or you'll hit a nasty NPE if the app is launched on a non test lab device. – guy.gc Oct 05 '20 at 09:02
2

Your 'device under test' should check the IP address and disable analytics if in the Firebase Test Lab IP range.

See my answer at https://stackoverflow.com/a/54954273/114549 for a Util method that handles this.

aaronvargas
  • 12,189
  • 3
  • 52
  • 52