15

I've been using var.test and bartlett.test to check basic ANOVA assumptions, among others, homoscedascity (homogeniety, equality of variances). Procedure is quite simple for One-Way ANOVA:

bartlett.test(x ~ g)  # where x is numeric, and g is a factor
var.test(x ~ g)

But, for 2x2 tables, i.e. Two-Way ANOVA's, I want to do something like this:

bartlett.test(x ~ c(g1, g2))  # or with list; see latter:
var.test(x ~ list(g1, g2))

Of course, ANOVA assumptions can be checked with graphical procedures, but what about "an arithmetic option"? Is that, at all, manageable? How do you test homoscedascity in Two-Way ANOVA?

aL3xa
  • 35,415
  • 18
  • 79
  • 112
  • I thought from the title that this question was completely made up. – sblom May 29 '10 at 06:04
  • What do you mean by "made up"? =) Homoscedascity (sphericity, homogeniety of variances) is, with normality of distribution in each subsample, one of basic ANOVA assumptions. Two-Way stands for ANOVA with two independent/factor variables. I accidentally posted this topic twice, so first post is closed as I recall... maybe because of that you thought this was made up/some kind of spam or so... =) – aL3xa May 29 '10 at 11:48

3 Answers3

18

Hypothesis testing is the wrong tool to use to asses the validity of model assumptions. If the sample size is small, you have no power to detect any variance differences, even if the variance differences are large. If you have a large sample size you have power to detect even the most trivial deviations from equal variance, so you will almost always reject the null. Simulation studies have shown that preliminary testing of model assumption leads to unreliable type I errors.

Looking at the residuals across all cells is a good indicator, or if your data are normal, you can use the AIC or BIC with/without equal variances as a selection procedure.

If you think there are unequal variances, drop the assumption with something like:

library(car)
model.lm <- lm(formula=x ~ g1 + g2 + g1*g2,data=dat,na.action=na.omit)
Anova(model.lm,type='II',white.adjust='hc3')

You don't loose much power with the robust method (hetroscedastic consistent covariance matrices), so if in doubt go robust.

Ian Fellows
  • 17,228
  • 10
  • 49
  • 63
  • Thorough, concise, skillfully written answer!!! I'd check residuals either way... but this answer was a breath of fresh air in my dogmatic understanding of statistics! Thank you so much Ian! – aL3xa May 29 '10 at 11:57
7

You can test for heteroscedasticity using the Fligner–Killeen test of homogeneity of variances. Supposing your model is something like

model<-aov(gain~diet*supplement)

fligner.test(gain~diet*supplement)

        Fligner-Killeen test of homogeneity of variances

data:  gain by diet by supplement 
Fligner-Killeen:med chi-squared = 2.0236, df = 2, p-value = 0.3636

You could have used bartlett.test (but this is more a test of non-normality than of equality of variances)

bartlett.test(gain~diet*supplement)
        Bartlett test of homogeneity of variances

data:  gain by diet by supplement 
Bartlett's K-squared = 2.2513, df = 2, p-value = 0.3244

Moreover, you could perform the Levene test for equal group variances in both one-way and two-way ANOVA. Implementations of Levene's test can be found in packages car (link fixed), s20x and lawstat

levene.test(gain~diet*supplement)  # car package version

Levene's Test for Homogeneity of Variance
      Df F value Pr(>F)
group 11  1.1034 0.3866
      36
Johan
  • 323
  • 4
  • 9
gd047
  • 29,749
  • 18
  • 107
  • 146
  • 4
    I don't think that's the right way of using `bartlett.test` when there are two factors. I believe it only uses the first variable. For example, if you run `bartlett.test(total_bill ~ sex*time, data=tips)` and `bartlett.test(total_bill ~ sex, data=tips)`, you get the same p-value, and df=1 for both. I think you need to use something like `bartlett.test(total_bill ~ interaction(time,sex), data=tips)`. In this case, df=3. – wch Apr 30 '11 at 06:25
6

For bartlett.test

bartlett.test(split(x,list(g1,g2)))

var.test is not applicable as it works only when there are two groups.

Jyotirmoy Bhattacharya
  • 9,317
  • 3
  • 29
  • 38