An experimental study on diversity for bagging and boosting with linear classifiers

LI Kuncheva, M Skurichina, RPW Duin

    Research output: Contribution to journalArticleScientificpeer-review

    115 Citations (Scopus)


    Recently bagging, boosting and the random subspace method have become popular combining techniques for improving weak classifiers. These techniques are designed for, and usually applied to, decision trees. In this paper, in contrast to a common opinion, we demonstrate that they may also be useful in linear discriminant analysis. Simulation studies, carried out for several artificial and real data sets, show that the performance of the combining techniques is strongly affected by the small sample size properties of the base classifier: boosting is useful for large training sample sizes, while bagging and the random subspace method are useful for critical training sample sizes. Finally, a table describing the possible usefulness of the combining techniques for linear classifiers is presented. Keywords: Bagging; Boosting; Combining classifiers; Linear classifiers; Random subspaces; Training sample size
    Original languageUndefined/Unknown
    Pages (from-to)245-258
    Number of pages14
    JournalInformation Fusion
    Issue number2
    Publication statusPublished - 2002


    • academic journal papers
    • ZX CWTS JFIS < 1.00

    Cite this