Effect Size Calculation on Conversion Rates of A/B Testing

Question

Trying to measure how likely an experiment group performed better than control group in conversion rate.

Business context: 1. Sample size: 80000, 2. Samples: 2, 3. Sample A converted: 1800, 4. Sample B converted: 1896, 5. Significant level: 0.05

I am trying to figure out how to get d to calculate power in qwr package in r:

Here is my code:

pwr.t.test(
      n=80000,
      d=???,
      sig.level = 0.05,
      power=NULL,
      type = "two.sample",
      alternative = "two.sided"
      )

Please let me know if you have any questions.

Thank you very much!

You need to give it a value for `power` for it to give you an effect size. At power of `0.8` the effect size is `0.014`. — mysteRious, Jul 27 '18 at 04:42
What if I want to calculate the power, how can I get the effect size number from observations? — Philip Liu, Jul 27 '18 at 13:01

Philip Liu · Answer 1 · 2018-07-27T13:36:24.103

0

h=ES.h(0.212,0.02)
#I think this will work.
#And use:
 pwr.p.test(
  n=80000,
  h=0.00844915,
  sig.level = 0.05,
  power=NULL,     
   alternative = "greater"
 )

This link accidentally answered my question: Sample size and power calculation in r as viable alternative to proc power in SAS?

edited Jul 27 '18 at 13:36

answered Jul 27 '18 at 13:10

Philip Liu

1
2

Effect Size Calculation on Conversion Rates of A/B Testing

1 Answers1