0

New to posting to Stack so apologies for any issues.

I'm learning to get more comfortable in R and currently looking at using broom/purr to run multiple stat tests at one time. An example of my current data looks like this:

Subject PreScoreTestA PostScoreTestA PreScoreTestB PostScoreTestB PreScoreTestC PostScoreTestC
1 30 40 6 8 12 10
2 15 12 9 13 7 7
3 20 22 11 12 9 10

But over many subjects and more tests. I want to do a dependent t-test to see scores changed over the course of a training program, but don't want to run a test for each score.

I've seen a couple examples of people using group by, nest, and map to run multiple t-tests, but their data was in a longer format

Is there a way to achieve the same goal while in a wide format? Or will I need to use pivot_longer to change the data.

Thanks in advance!

ETA had an edit here but was giving incorrect results and so have removed Still looking for some help on the arguments and same length

ETA Version 2

I did find a workaround using pairwise.t.test (code below). It gives the same p-values as doing t.test across individual assessments. I'm curious why it'd be working for pairwise.t.test but not t.test. Please let me know if anyone was any ideas!

    results <- testb %>%
     pivot_longer(-Subject, 
                   names_to = c("time", "test"), values_to = "score", 
                   names_pattern = "(Pre|Post)(.*)") %>%
     group_by(test) %>% 
     nest() %>% 
     mutate(ttests = map(.x=data, ~tidy(pairwise.t.test(.x$score, .x$time, paired = TRUE, p.adjust.method = "none")))) %>%  
     unnest(ttests)
TibialCuriosity
  • 47
  • 5
  • 1
    Most likely you will need to pivot the data because this is not in a tidy format. What exactly is the hypothesis you are testing? Are you trying to generate a t-test comparison for each of the different tests (ie, TestA, TestB, TestC)? – MrFlick Nov 13 '21 at 05:52
  • @MrFlick Thanks for your reply, any tips on pivoting the data? I'm trying to figure it out but running into some difficulty. Each test is a fitness assessment. The hypothesis being that after a period of training subjects fitness would significantly improve. And yes it would be a t-test comparison for each assessment – TibialCuriosity Nov 13 '21 at 06:01

2 Answers2

3

Yes, some pivoting is needed. Asssuming you have no directional hypotheses and you want to do a pre-post assessment for each test, this might be what you are looking for:

df <- as.data.frame(rbind(c(1,  30, 40, 6,  8,  12, 10),
                          c(2,  15, 12, 9,  13, 7,  7),
                          c(3,  20, 22, 11, 12, 9,  10)))

names(df) <- c("Subject",   
               "PrePushup", "PostPushup",   
               "PreRun",    "PostRun",  
               "PreJump",   "PostJump")

df %>% 
  pivot_longer(-Subject, 
               names_to = c("time", "test"), values_to = "score", 
               names_pattern = "(Pre|Post)(.*)") %>% 
  group_by(test) %>% 
  nest() %>% 
  mutate(t_tests = map(data, ~t.test(score ~ time, data = .x, paired = TRUE))) %>% 
  pull(t_tests) %>% 
  purrr::set_names(c("Pushup", "Run", "Jump"))

$Pushup

    Paired t-test

data:  score by time
t = 0.79241, df = 2, p-value = 0.5112
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -13.28958  19.28958
sample estimates:
mean of the differences 
                      3 


$Run

    Paired t-test

data:  score by time
t = 2.6458, df = 2, p-value = 0.1181
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -1.461250  6.127916
sample estimates:
mean of the differences 
               2.333333 


$Jump

    Paired t-test

data:  score by time
t = -0.37796, df = 2, p-value = 0.7418
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -4.127916  3.461250
sample estimates:
mean of the differences 
             -0.3333333
huttoncp
  • 161
  • 4
  • Thank you for this, I think it has helped a lot, though I may have some questions on what the code signifies. But I'm running into an issue I didn't think of. My columns are named a bit differently as follows: PrePushup, PostPushup, PreRun, PostRun etc. I don't think name_sep works in this or I'm doing it incorrectly. I've been having a look at name_pattern but struggling as well. Any advice on this? Apologies I did not think the naming would matter that much. Thank you again – TibialCuriosity Nov 13 '21 at 07:03
  • 1
    the names_pattern argument uses a regular expression to separate the columns. You can learn more about regex in R from this blog post I wrote a while back: https://craig.rbind.io/post/2020-06-28-asgr-2-3-string-manipulation/. In this case, you could try something like names_pattern = "(Pre|Post)(.*)", where we split into 2 components delineated by the parentheses, where the 1st one that will go to the new "time" column can have a value of "Pre" or ("|") "Post" and the other component is the remainder of the variable name that will go into the new "test" column. – huttoncp Nov 13 '21 at 07:12
  • 1
    I've just updated my answer to use names_pattern instead of names_sep to make it more flexible – huttoncp Nov 13 '21 at 07:19
  • Thanks again, I've marked it as solved because it has answered my original question. However, I'm running into an error of not all arguments have the same length. Error occurred in group 1 t = "1.5". I know a paired t-test requires the same number of observations, but I can run a paired t-test outside of this function on just the 1.5 assessment and it is fine. Does every single variable need to have the same length when mapping functions? Or is there another issue? – TibialCuriosity Nov 13 '21 at 09:33
  • I've been doing some more searching and running into similar issues. Either I'm getting that same error (which doesn't make sense since it should work as long as length is same within each assessment and I'm doing na.exclude or I can use x,y instead of x~y but then I get to a non~numeric operator error. I've fixed this by recoding time to 1 and 2 instead of pre and post but then it looks like the t-test is comparing the means of the assessments to the mean of 1 and 2, or 1.5 – TibialCuriosity Nov 13 '21 at 14:09
  • 1
    The loop portion executed by purrr::map() doesn't require the same length because it returns a list as output. The code is failing because the t.test() function is throwing an error on the pair which has a missing value for one of the original measurement columns. You can get a loop or a map function to continue running even if one item fails by wrapping the function you're applying to each item in try(), e.g. map(data, ~try(t.test(score ~ time, data = .x, paired = TRUE))) – huttoncp Nov 14 '21 at 09:33
  • 1
    If you have imbalanced or incomplete data for repeated measures data, you might want to consider examining it using a mixed effects model rather than a paired t-test. However, this topic falls beyond the scope of stack overflow and is more appropriately explored via stats.stackexchange. This thread may be helpful: https://stats.stackexchange.com/a/93959 or this one: https://stats.stackexchange.com/a/310424 – huttoncp Nov 14 '21 at 09:41
2

Here is a try without pivoting into long format: This again was finished with the help of the incredible akrun! See here: How to apply t.test() to multiple pairs of columns after mutate across:

df %>%
  summarise(across(starts_with('PreScore'), ~  t.test(.,
                                                      get(str_replace(cur_column(), "^PreScore", "PostScore")))$p.value, 
                   .names = "{.col}_TTest"))

  PreScoreTestA_TTest PreScoreTestB_TTest PreScoreTestC_TTest
1            0.767827            0.330604           0.8604162
TarJae
  • 72,363
  • 6
  • 19
  • 66
  • Thanks for this! I'll have a read of that post cause I don't understand some of this code. Is there a reason why the p-values are different than the solution above? I tried running it on my data and it did work, but the p-values would be different compared to running t-tests individually – TibialCuriosity Nov 13 '21 at 23:39
  • I've tried to add a comment on the post you linked asking akrun, but don't have enough reputation. Just to expand on the previous comment, if I run the following code `t.test(df$PostScoreTestA, df$PreScoreTestB, data = df, paired = TRUE)`, I get a p-value of 0.2272 (0.2136 if I switch post and pre order) compared to the 0.7678 in your output above – TibialCuriosity Nov 14 '21 at 00:17
  • 1
    The discrepancy is because this solution doesn't specify that they are paired-sample t-tests by setting the paired argument of t.test to TRUE. – huttoncp Nov 14 '21 at 09:26