Abstract
Researchers occasionally have to work with an extremely small sample size, defined herein as N ≤ 5. Some methodologists have cautioned against using the ttest when the sample size is extremely small, whereas others have suggested that using the ttest is feasible in such a case. The present simulation study estimated the Type I error rate and statistical power of the one and twosample ttests for normally distributed populations and for various distortions such as unequal sample sizes, unequal variances, the combination of unequal sample sizes and unequal variances, and a lognormal population distribution. Ns per group were varied between 2 and 5. Results show that the ttest provides Type I error rates close to the 5% nominal value in most of the cases, and that acceptable power (i.e., 80%) is reached only if the effect size is very large. This study also investigated the behavior of the Welch test and a ranktransformation prior to conducting the ttest (ttestR). Compared to the regular ttest, the Welch test tends to reduce statistical power and the ttestR yields false positive rates that deviate from 5%. This study further shows that a paired ttest is feasible with extremely small Ns if the withinpair correlation is high. It is concluded that there are no principal objections to using a ttest with Ns as small as 2. A final cautionary note is made on the credibility of research findings when sample sizes are small.
Original language  English 

Number of pages  12 
Journal  Practical Assessment, Research & Evaluation 
Volume  18 
Issue number  10 
Publication status  Published  2013 
Fingerprint
Dive into the research topics of 'Using the Student’s ttest with extremely small sample sizes'. Together they form a unique fingerprint.Datasets

Supplementary material for the paper: Using the Student’s ttest with extremely small sample sizes
de Winter, J. C. F. (Creator), TU Delft  4TU.ResearchData, 2019
DOI: 10.4121/UUID:7BA3F2CB7FF645318AE4D44462DADEF5
Dataset/Software: Dataset