4

I'm running an A/B test in Firebase. The client is getting values and responding appropriately. I can see "current users" in the live view:

Live view showing users

But even after 48 hours I still see "Running (0 users exposed)" in the headline of the experiment.

How can these be "active experiement users" yet not show up as part of the experiment?

For comparison, here's the staging build of the app, which has assigned users with the same code, just building with a different scheme and hitting a different Firebase app:

Test setup showing users added to experiment

We call fetchAndActivate on launch:

FirebaseApp.configure()
RemoteConfig.remoteConfig().fetchAndActivate { _, error in
    print(error ?? "Fetched!")
}
buildsucceeded
  • 4,203
  • 4
  • 34
  • 72

1 Answers1

1

For now, it happens (no data in Firebase remote config A/B test experiment) if you have an activation event configured for A/B test experiment.

Check these:

  • If you have 2 different experiments, both will fail to get results even if you have "activation event" configured only in 1 of them. Additionally, remote config will not work as well, you'll be able to get only default values.

  • it takes many hours before you can see any data in your experiment, wait for at least 24 hours. It will show 0 for many hours after the start.

  • make sure your users have the version of your app with the latest SDK.

  • Since your experiment is with Remote Config: When activateFetched() is called, all events from that point on will be tagged with the experiment. If you have a goal or activation event that happens before activateFetched(), such as automatic events like first_open, session_start, etc., the experiment setup might be wrong.

  • Make sure to call fetch() and activateFetched() before the activation event occurs.

  • The good way to check if your experiment is working now is to set it to a specific version you didn't publish yet and check logs from remote config with the fresh app install(or erase all app data & restart). It should show different variant every time you reinstall the app, since your Firebase Instance ID changes after app reinstall/app data erase. If you see variants change - then A/B test is running well.

bArmageddon
  • 8,088
  • 6
  • 22
  • 40