Abstract
Researchers occasionally have to work with an extremely small sample size, defined herein as N ≤ 5. Some methodologists have cautioned against using the t-test when the sample size is extremely small, whereas others have suggested that using the t-test is feasible in such a case. The present simulation study estimated the Type I error rate and statistical power of the one- and two-sample t-tests for normally distributed populations and for various distortions such as unequal sample sizes, unequal variances, the combination of unequal sample sizes and unequal variances, and a lognormal population distribution. Ns per group were varied between 2 and 5. Results show that the t-test provides Type I error rates close to the 5% nominal value in most of the cases, and that acceptable power (i.e., 80%) is reached only if the effect size is very large. This study also investigated the behavior of the Welch test and a rank-transformation prior to conducting the t-test (t-testR). Compared to the regular t-test, the Welch test tends to reduce statistical power and the t-testR yields false positive rates that deviate from 5%. This study further shows that a paired t-test is feasible with extremely small Ns if the within-pair correlation is high. It is concluded that there are no principal objections to using a t-test with Ns as small as 2. A final cautionary note is made on the credibility of research findings when sample sizes are small.
Original language | English |
---|---|
Number of pages | 12 |
Journal | Practical Assessment, Research & Evaluation |
Volume | 18 |
Issue number | 10 |
Publication status | Published - 2013 |
Fingerprint
Dive into the research topics of 'Using the Student’s t-test with extremely small sample sizes'. Together they form a unique fingerprint.Datasets
-
Supplementary material for the paper: Using the Student’s t-test with extremely small sample sizes
de Winter, J. C. F. (Creator), TU Delft - 4TU.ResearchData, 9 Sept 2019
DOI: 10.4121/UUID:7BA3F2CB-7FF6-4531-8AE4-D44462DADEF5
Dataset/Software: Dataset