| Attribute | ASD (n = 76) | TD (n = 78) |
|---|---|---|
| Age: mean (SD) years | 5.6 (1.7) | 5.7 (2.0) |
Multiple testing corrections
PSTAT197A/CMPSC190DD Fall 2022
FWER Example
If individual tests are exactly controlled at \(\alpha = 0.05\) and independent, at least one error is nearly certain by 100 tests.
Familywise error rate as a function of the number of tests, assuming tests are independent with exact type I error 0.05.
Computations
trim_fn <- function(x){
x[x > 3] <- 3
x[x < -3] <- -3
return(x)
}
asd_clean <- asd %>%
select(-ados) %>%
# log transform
mutate(across(.cols = -group, log10)) %>%
# center and scale
mutate(across(.cols = -group, ~ scale(.x)[, 1])) %>%
# trim outliers (affects results??)
mutate(across(.cols = -group, trim_fn))
asd_nested <- asd_clean %>%
pivot_longer(-group,
names_to = 'protein',
values_to = 'level') %>%
nest(data = c(level, group))
asd_nested %>% head(4)# A tibble: 4 × 2
protein data
<chr> <list>
1 CHIP <tibble [154 × 2]>
2 CEBPB <tibble [154 × 2]>
3 NSE <tibble [154 × 2]>
4 PIAS4 <tibble [154 × 2]>
# compute for several groups
test_fn <- function(.df){
t_test(.df,
formula = level ~ group,
order = c('ASD', 'TD'),
alternative = 'two-sided',
var.equal = F)
}
tt_out <- asd_nested %>%
mutate(ttest = map(data, test_fn)) %>%
unnest(ttest) %>%
arrange(p_value)
tt_out %>% head(5)# A tibble: 5 × 9
protein data statistic t_df p_value alternative estimate lower_ci
<chr> <list> <dbl> <dbl> <dbl> <chr> <dbl> <dbl>
1 DERM <tibble> -6.10 151. 8.27e-9 two.sided -0.885 -1.17
2 RELT <tibble> -5.65 152. 7.82e-8 two.sided -0.775 -1.05
3 FSTL1 <tibble> -5.27 152. 4.66e-7 two.sided -0.783 -1.08
4 C1QR1 <tibble> -5.26 152. 4.79e-7 two.sided -0.782 -1.08
5 Calcineurin <tibble> -5.24 151. 5.37e-7 two.sided -0.734 -1.01
# … with 1 more variable: upper_ci <dbl># multiple testing corrections
m <- nrow(tt_out)
hm <- log(m) + 1/(2*m) - digamma(1)
tt_corrected <- tt_out %>%
select(data, protein, p_value) %>%
mutate(rank = row_number()) %>%
mutate(p_bh = p_value*m/rank,
p_by = p_value*m*hm/rank,
p_bonf = p_value*m)
tt_corrected %>% head(5)# A tibble: 5 × 7
data protein p_value rank p_bh p_by p_bonf
<list> <chr> <dbl> <int> <dbl> <dbl> <dbl>
1 <tibble [154 × 2]> DERM 0.00000000827 1 0.0000109 0.0000845 1.09e-5
2 <tibble [154 × 2]> RELT 0.0000000782 2 0.0000515 0.000400 1.03e-4
3 <tibble [154 × 2]> FSTL1 0.000000466 3 0.000205 0.00159 6.14e-4
4 <tibble [154 × 2]> C1QR1 0.000000479 4 0.000158 0.00122 6.31e-4
5 <tibble [154 × 2]> Calcineurin 0.000000537 5 0.000141 0.00110 7.07e-4Results
# A tibble: 10 × 2
protein p_by
<chr> <dbl>
1 DERM 0.0000845
2 RELT 0.000400
3 Calcineurin 0.00110
4 C1QR1 0.00122
5 MRC2 0.00132
6 IgD 0.00136
7 CXCL16, soluble 0.00149
8 PTN 0.00154
9 FSTL1 0.00159
10 Cadherin-5 0.00179 Neat graphic: volcano plot
Upregulation and downregulation of serum levels of proteins analyzed – p-values against number of doublings (positive) or halvings (negative) of serum level in ASD group relative to TD group.