3

I have been looking around and I am quite confused about Tukey adjustment in emmeans. The documentation of emmeans doesn't mention Tukey HSD at all, but in here it is said that "For most contrast() results, adjust is often something else, depending on what type of contrasts are created. For example, pairwise comparisons default to adjust = "tukey", i.e., the Tukey HSD method.".

As I understand, Tukey HSD is essentially a series of pairwise t-test with adjustment for type I error. But the emmeans function is calculating estimated marginal means (EMMs), which I assume are not pairwise t-tests; then applying the Tukey adjustment to emmeans output, would not be an equivalent to Tukey HSD post hoc test.

A second related question would be what the function "tukey.emmc", also from emmeans, does?

[Update] I guess my second question is, what is the difference between tukey.emmc and contrast() with 'adjust = "tukey"'?

zx8754
  • 52,746
  • 12
  • 114
  • 209
jordi
  • 91
  • 1
  • 7
  • People who bother to answer questions would like to see some evidence that you have at least read it. – Russ Lenth Nov 30 '20 at 01:12
  • I have indeed read your answer and upvoted it. Unfortunately, my reputation is still too low for my votes to be visible. Besides that, from your answer I understood that my second question was not properly formulated, so I added an update. – jordi Dec 01 '20 at 06:16
  • I added to my answer. – Russ Lenth Dec 01 '20 at 17:22

1 Answers1

5

Using adjust = "tukey" means that critical values and adjusted P values are obtained from the Studentized range distribution qtukey() and ptukey() respectively. Those are the same critical values that are used in the Tukey HSD test. But to put a very fine edge on it, the Tukey HSD method is really defined only for independent samples of equal size, which may or may not be the case for emmeans() results. For more details, see ? summary.emmGrid and refer to the section on P-value adjustments.

Regarding the second question, both pairwise.emmc() generates contrast coefficients for pairwise comparisons; as does revpairwise.emmc(). Here is a third possibility:

> emmeans:::tukey.emmc
function(levs, reverse = FALSE, ...) {
    if (reverse)
        revpairwise.emmc(levs, ...)
    else
        pairwise.emmc(levs, ...)
}

That is, tukey.emmc() invokes one of those pairwise-comparison methods depending on reverse. Thus, contrast(..., method = "tukey", reverse = TRUE) is equivalent to contrast(..., method = "revpairwise").

Every .emmc function passes a default adjustment method to contrast(), and in the case of pairwise.emmc() and tukey.emmc(), that default is adjust = "tukey". Thus, calling contrast(..., method = "pairwise") is the same as contrast(..., method = "pairwise", adjust = "tukey"). Whereas calling some other contrast function may produce different defaults. For example, consec.emmc() passes the "mvt" adjustment by default:

> emmeans:::consec.emmc(1:4)
  2 - 1 3 - 2 4 - 3
1    -1     0     0
2     1    -1     0
3     0     1    -1
4     0     0     1

> attributes(.Last.value)
$names
[1] "2 - 1" "3 - 2" "4 - 3"

$row.names
[1] 1 2 3 4

$class
[1] "data.frame"

$desc
[1] "changes between consecutive levels"

$adjust
[1] "mvt"

An additional comment about the Tukey adjustment: That adjustment is only appropriate for a single set of pairwise comparisons. If you specify adjust = "tukey" for non-pairwise comparisons or arbitrary contrasts, it will overrule you and use the "sidak" adjustment instead.

Russ Lenth
  • 5,922
  • 2
  • 13
  • 21