NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Back to Results
Bayes Error Rate Estimation Using Classifier EnsemblesThe Bayes error rate gives a statistical lower bound on the error achievable for a given classification problem and the associated choice of features. By reliably estimating th is rate, one can assess the usefulness of the feature set that is being used for classification. Moreover, by comparing the accuracy achieved by a given classifier with the Bayes rate, one can quantify how effective that classifier is. Classical approaches for estimating or finding bounds for the Bayes error, in general, yield rather weak results for small sample sizes; unless the problem has some simple characteristics, such as Gaussian class-conditional likelihoods. This article shows how the outputs of a classifier ensemble can be used to provide reliable and easily obtainable estimates of the Bayes error with negligible extra computation. Three methods of varying sophistication are described. First, we present a framework that estimates the Bayes error when multiple classifiers, each providing an estimate of the a posteriori class probabilities, a recombined through averaging. Second, we bolster this approach by adding an information theoretic measure of output correlation to the estimate. Finally, we discuss a more general method that just looks at the class labels indicated by ensem ble members and provides error estimates based on the disagreements among classifiers. The methods are illustrated for artificial data, a difficult four-class problem involving underwater acoustic data, and two problems from the Problem benchmarks. For data sets with known Bayes error, the combiner-based methods introduced in this article outperform existing methods. The estimates obtained by the proposed methods also seem quite reliable for the real-life data sets for which the true Bayes rates are unknown.
Document ID
20130011282
Acquisition Source
Ames Research Center
Document Type
Reprint (Version printed in journal)
External Source(s)
Authors
Tumer, Kagan
(NASA Ames Research Center Moffett Field, CA, United States)
Ghosh, Joydeep
(Texas Univ. Austin, TX, United States)
Date Acquired
August 27, 2013
Publication Date
January 1, 2003
Publication Information
Publication: International Journal of Smart Engineering System Design
Volume: 5
Issue: 2
Subject Category
Engineering (General)
Distribution Limits
Public
Copyright
Other

Available Downloads

There are no available downloads for this record.
No Preview Available