Kathryn Hoisington-Shaw
Department of Psychology
The Ohio State University
Title: There and Back Again: A Sequential Testing Tale
Abstract: Planning a study requires researchers to make a number of decisions, not the least of which is what sample size to choose. The gold standard in psychological research is to run a power analysis to calculate sample size, which requires using an effect size as an input in this equation. Results of a recent meta-science study show that the most common type of effect size used in these power analyses are sourced from previously collected data. However, use of these point estimated effect sizes do not account for sampling variability and result in estimates that are imprecise. There are modern power methods that account for this sampling variability, but Monte Carlo simulations show that these approaches often result in unrealistically high sample sizes. Due to these limitations, this talk will move away from using statistical power, and will focus on the potential of using the sequential probability ratio test (SPRT) for study design and analysis instead. How to use SPRT in practice will be discussed, as well as the proposal of two new extended SPRT approaches that take the sampling variability of estimates into account when using previously collected data to plan future research. The results of Monte Carlo simulations to evaluate and compare these methods will be discussed, as well as a comparison to the modern power approaches, and final recommendations for study design will be considered.
Discussant: Jingdan (Diana) Zhu