Sample size and power calculation in r as viable alternative to proc power in SAS?

Question

So I am trying to see how close the sample size calculations (for two sample independent proportions with unequal samples sizes) are between proc power in SAS and some sample size functions in r. I am using the data found here at a UCLA website.

The UCLA site gives parameters as follows:

p1=.3,p2=.15,power=.8,null difference=0, and for the two-sided tests it assumes equal sample sizes;

for the unequal sample size tests the parameters are the same, with group weights of 1 for group1 and 2 for group2, and the tests they perform are one-sided.

I am using the r function

pwr.t.test(n=NULL,d=0,sig.level=0.05,type="two.sample",alternative="two.sided")

from the pwr package.

So if I input the parameter selections as the UCLA site has for their first example, I get the following error:

Error in uniroot(function(n) eval(p.body) - power, c(2, 1e+07)) :
  f() values at end points not of opposite sign.

This appears to be because the difference is undetectable by r. I set d=.5 and it ran. Would SAS give error as well for too small difference? It doesn't in the example as their null difference is zero also.

I also get the error above when using

pwr.2p.test(h = 0, n = , sig.level =.05, power = .8)

and

pwr.chisq.test(w =0, N = , df =1 , sig.level =.05, power =.8 ).

I may be doing something horribly wrong, but I cant seem to really find a way if the hypothesized difference is 0.

I understand that SAS and r are using different methods for calculating the power, so I shouldn't expect to get the same result. I am really just trying to see if I can replicate proc power results in r.

I have been able to get near identical results for the first example with equal sample sizes and a two-sided alternative using

 bsamsize(p1=.30,p2=.15,fraction=.5, alpha=.05, power=.8)

from the Hmisc package. But when they do 1-sided tests with unequal sample sizes I can't replicate those.

Is there a way to replicate the process in r for the 1-sided sample size calculations for unequal group sizes?

Cheers.

I'm a little confused why you're using `pwr.t.test` for a power analysis of proportions as opposed to say `pwr.chisq.test`? Can you be slightly more specific (e.g. show the specific code you ran rather than just "I input the parameter sections as the UCLA site has for their first example") ? — Ben Bolker, Mar 13 '13 at 20:29
Hi. I added some more info above. Please let me know if that doesn't help. Also, the lines of code above the exact code I ran in r, with the UCLA values in the function. — user27008, Mar 13 '13 at 20:43
My impression is that `d > 0` is necessary for `pwr.t.test` to give meaningful output. — Blue Magister, Mar 13 '13 at 21:01

Blue Magister · Accepted Answer · 2014-11-25T16:42:11.910

3

In pwr.t.test and its derivatives, d is not the null difference (that's assumed to be zero), but the effect size/hypothesized difference between the two populations. If the difference between population means is zero, no sample size will let you detect a nonexistent difference.

If population A has a proportion of 15% and population B has a proportion of 30%, then you use the function pwr::ES.h to calculate the effect size and do a test of proportions like:

> pwr.2p.test(h=ES.h(0.30,0.15),power=0.80,sig.level=0.05)

     Difference of proportion power calculation for binomial distribution (arcsine transformation) 

              h = 0.3638807
              n = 118.5547
      sig.level = 0.05
          power = 0.8
    alternative = two.sided

NOTE: same sample sizes

> pwr.chisq.test(w=ES.w1(0.3,0.15),df=1,sig.level=0.05,power=0.80)

     Chi squared power calculation 

              w = 0.2738613
              N = 104.6515
             df = 1
      sig.level = 0.05
          power = 0.8

NOTE: N is the number of observations

edited Nov 25 '14 at 16:42

answered Mar 14 '13 at 14:14

Blue Magister

13,044
5
38
56

Ah, okay. Well, for using `pwr.2p.test(h = .15, n =, sig.level =.05, power =.8 ,alternative="two.sided")`, I got n=697.6765 per group. Ucla got n=121,120,and 132 for there runs of three different methods. For using `pwr.chisq.test(w=0.15,df=1,sig.level=0.05,power=0.80)`, I got N= 348.8382. I am still not understanding why I am getting such vast differences in n's. – user27008 Mar 14 '13 at 14:35
2

So I ended up being able to replicate the results by performing the calculations by hand. I think it would be worthwhile to make a package in r to do some of the tests that proc power performs. I have my formulas set, but am not sure how difficult it is to make a package that would carry out the calculations necessary. – user27008 Mar 14 '13 at 22:51
I think your interpretation of h parameter is wrong @Blue Magister because it is possible to deliver h higher than 1, check this example pwr.2p.test(h=2, sig.level = 0.05, power = 0.8). By the way h stands for th Cohen's d that is reffered to belong to interval (-3,3) Check info about this package [here](http://www.statmethods.net/stats/power.html) – Marcin Nov 08 '14 at 19:07
By the way maybe you'll know the answer for this similar question http://stats.stackexchange.com/questions/123111/cohens-d-in-proportion-test ? – Marcin Nov 08 '14 at 19:13
1

Reviewing this answer I realized I forgot to use the effect size functions `pwr::ES.h`, `pwr::ES.w1`, and `pwr::ES.w2`. The question has been edited to reflect that. – Blue Magister Nov 25 '14 at 16:43

Sample size and power calculation in r as viable alternative to proc power in SAS?

1 Answers1

Linked