4

I am trying to limit my A/B test to the versions of my iOS app starting from 11.3.10. The reason is that older versions would behave identically for all variants as they contain no knowledge of parameters involved. This, in turn, would introduce noise which would make my experiment less conclusive.

So, I was adding the following regex to my app version property:

1[1-9]\.[3-9]\.[1-9][0-9]

Unfortunately, it didn't work. No users were taking part in my experiment. I then decided to try 1[1-9]\.[3-9]\.[1-9][0-9].*. But still, no luck. If I omit this setting and run my experiment, I immediately see the data for users taking part in it. 138 in 30 minutes to be precise.

My latest version (11.3.10) has roughly 30% adoption, so clearly I would see some users if the setting was correct.

I will be grateful if you could let me know what I am doing wrong here.

Andriy Gordiychuk
  • 6,163
  • 1
  • 24
  • 59

2 Answers2

11

I got in touch with Firebase support and figured out what was wrong.

Although their docs mention "Version" everywhere (https://firebase.google.com/docs/ab-testing/abtest-config), the reality is that they are looking for Build Number instead. So, in my case, I needed to target all "versions" greater than 379 rather than all "versions" greater than 11.3.11.

If anyone is curious, here is a regex which achieves that:

(\d{4,}|379|3[8-9][0-9]|[4-9][0-9][0-9])

Andriy Gordiychuk
  • 6,163
  • 1
  • 24
  • 59
  • 1
    LIFE SAVER. Can't believe they didn't bother to say that in the documentation. I lost too much time trying to figure out what was wrong with my A/B tests. – tperei Jan 23 '20 at 21:55
1

For 379+, these expressions might also work then,

^\d{4,}|379|3[8-9]\d|[4-9]\d{2}$

\d{4,}|379|3[8-9]\d|[4-9]\d{2}

^[0-9]{4,}|379|3[8-9][0-9]|[4-9][0-9]{2}$

[0-9]{4,}|379|3[8-9][0-9]|[4-9][0-9]{2}

\b[0-9]{4,}|379|3[8-9][0-9]|[4-9][0-9]{2}\b

If you wish to explore/simplify/modify the expression, it's been explained on the top right panel of regex101.com. If you'd like, you can also watch in this link, how it would match against some sample inputs.


Demo 2

Community
  • 1
  • 1
Emma
  • 27,428
  • 11
  • 44
  • 69
  • 1
    Sorry, I made a typo! It wasn't [1-0] it was [1-9]. I corrected the mistake now. – Andriy Gordiychuk Aug 10 '19 at 16:26
  • 1
    the problem is that Firebase does not give me details on which app versions it decided to include/omit. So, I do not have sample inputs from the real world. Trying my test cases on regex101.com leads me to believe that my regex is correct. I think that Firebase just requires some extra setup. However, the docs only mention that regex needs to comply with RE2 standard. And I think mine does comply. – Andriy Gordiychuk Aug 10 '19 at 16:35