4

As some other questions pointed out if you're setting up a Remote Config based AB test there's no activation event based on user first opened.

We want to AB test our new onboarding flow against the previous onboarding experience however without a startup trigger we're not sure how to properly create this experiment.

One SO answer talks about sending a custom activation event with a timestamp and then filtering the test participants by that timestamp e.g. custom_first_open > 1234567... however the onboarding flow is the first thing the user is to see.

From my understanding as soon as the user initializes their remote config they will be subscribed to any active experiments. We would have to send the custom event before initialization and it would have to be immediately available to the AB test. AB test data and Firebase events both seem to be very slow to register (hours to days) so I doubt it would properly configure the user for the onboarding test using this trick.

Is there another way to use AB testing to test onboarding efficacy only against new users?

alexgophermix
  • 4,189
  • 5
  • 32
  • 59

1 Answers1

0

There are a couple of ways to go about this.

First, when you create the experiment you can limit the experiment targeting to only include users on a new version or build of the app -- or country, etc. example targeting

You can also only target users in an Audience you define, which give you pretty flexible abilities to define whatever group you'd like to roll the tests out to. creating an audience

Note - we tend to recommend you use first_touch_timestamp for correctly identifying new users. Better than first_open.

Also, outcomes are easier to measure when you're looking at ARPU/LTV outcomes

Adam Landis
  • 1