0

I am looking for a way to calculate the observed power of an AB test, in javascript but can't figure out the formula.

Here is a live example : https://abtestguide.com/calc/

screenshot : observed power on abtestguide calc

I managed to reproduce all the field calculations except the observed power.

My research led me to a few articles that share the formula in R, but I have no idea how to interpret that in JS.

I expect the output of the formula to be the same as the one obtained on abtestguide's calculator.

Thank you very much in advance to anyone that will help, it's appreciated.

PS : Here is an example I found of the formula in R, if it helps : http://daniellakens.blogspot.com/2014/12/observed-power-and-what-to-do-if-your.html

Vlom31
  • 13
  • 2
  • FYI: `N<-33` does not `#produce 100 simulated participants`. – r2evans Sep 19 '19 at 04:21
  • You're asking somebody here to translate into JS not just the R code here, but also the requisite logic in the `MBESS` and `pwr` packages? Good luck! – r2evans Sep 19 '19 at 04:23
  • I disagree with the no attempt, I have attempted quite a bit actually with no success. Sorry if the question is still unclear, this is my first time asking on SO, so I guess thanx for the warm welcome man. – Vlom31 Sep 19 '19 at 05:46
  • 1
    I have edited the question, the R part was just to help as that's the closest thing I found during my research, but if you know the formula directly in JS please, do share. – Vlom31 Sep 19 '19 at 05:52
  • Vlom31: my initial vote wasn't out of malice, and perhaps a little misunderstanding of your needs (either my-bad for mis-reading, or just a little mis-communication). I'm glad you got an answer. – r2evans Sep 19 '19 at 14:48
  • No problem @r2evans I agree the initial question was a bit confusing and as it turned out the R portion was unnecessary, thanx for pointing that out. – Vlom31 Sep 20 '19 at 04:22

1 Answers1

0

Very simple to find source ;-)

You can inspect element in Chrome, put breakpoint on subtree change and check call stack.

There is a lot of jQuery, but then somewhere in the middle file:

https://abtestguide.com/calc/js/script_fp.js
Setting is performed on line 230(?):
$('#power').html(perc(power));

And you can find power formulas, etc. somewhere before. In case you are unsure, put breakpoint somewhere in this file and check what is going on - there are also some ifs, so formula is a bit cyclomatic complex.

Hope enough to play with ;-)

Jan
  • 2,178
  • 3
  • 14
  • 26
  • Thanx a ton @Tom it was in there, I got intimidated by all the jquery the first time I took a peak into that file so I went on and recreated all the formulas myself. For future reference : power = 1 - normDist((( crB+seB*zCritical - crA)/seA), 0, 1, true); the variables are self-explanatory. – Vlom31 Sep 19 '19 at 08:31
  • I also dislike jQuery for this - still some trafic, in case you need to stop after click, there is an easy way mentioned here https://stackoverflow.com/questions/18877190/chrome-debugging-break-on-next-click-event/57784373#57784373 – Jan Sep 19 '19 at 09:32
  • Thanx for that debug function Tom, I added it to my dash snippet collection :) – Vlom31 Sep 19 '19 at 09:48
  • As I answer mostly interesting question for fun, thinking about organizing answers, but probably outside as there are no options(?) or others do not like constructive activity(?). Also have some web related on https://github.com/eltomjan/ETEhomeTools/tree/master/HTM_HTA but just small codes. – Jan Sep 19 '19 at 09:53