TY - GEN
T1 - Combining accuracy and prior sensitivity for classifier design under prior uncertainty
AU - Landgrebe, TCW
AU - Duin, RPW
PY - 2006
Y1 - 2006
N2 - Consideringtheclassi¿cationprobleminwhichclasspriorsormisallocationcostsarenotknownprecisely,receiveroperatorcharacteristic(ROC)analysishasbecomeastandardtoolinpatternrecognitionforobtainingintegratedperformancemeasurestocopewiththeuncertainty.Similarly,insituationsinwhichpriorsmayvaryinapplication,theROCcanbeusedtoinspectperformanceovertheexpectedrangeofvariation.InthispaperwearguethateventhoughmeasuressuchastheareaundertheROC(AUC)areusefulinobtaininganintegratedperformancemeasureindependentofthepriors,itmayalsobeimportanttoincorporatethesensitivityacrosstheexpectedprior-range.Weshowthataclassi¿ermayresultinagoodAUCscore,butapoor(large)priorsensitivity,whichmaybeundesirable.Amethodologyisproposedthatcombinesbothaccuracyandsensitivity,providinganewmodelselectioncriterionthatisrelevanttocertainproblems.Experimentsshowthatincorporatingsensitivityisveryimportantinsomerealisticscenarios,leadingtobettermodelselectioninsomecases
AB - Consideringtheclassi¿cationprobleminwhichclasspriorsormisallocationcostsarenotknownprecisely,receiveroperatorcharacteristic(ROC)analysishasbecomeastandardtoolinpatternrecognitionforobtainingintegratedperformancemeasurestocopewiththeuncertainty.Similarly,insituationsinwhichpriorsmayvaryinapplication,theROCcanbeusedtoinspectperformanceovertheexpectedrangeofvariation.InthispaperwearguethateventhoughmeasuressuchastheareaundertheROC(AUC)areusefulinobtaininganintegratedperformancemeasureindependentofthepriors,itmayalsobeimportanttoincorporatethesensitivityacrosstheexpectedprior-range.Weshowthataclassi¿ermayresultinagoodAUCscore,butapoor(large)priorsensitivity,whichmaybeundesirable.Amethodologyisproposedthatcombinesbothaccuracyandsensitivity,providinganewmodelselectioncriterionthatisrelevanttocertainproblems.Experimentsshowthatincorporatingsensitivityisveryimportantinsomerealisticscenarios,leadingtobettermodelselectioninsomecases
KW - conference contrib. refereed
KW - CWTS 0.75 <= JFIS < 2.00
UR - http://www.springerlink.com/content/j7411g54780230j7/fulltext.pdf
M3 - Conference contribution
SN - 3-540-37236-9
SP - 512
EP - 521
BT - Structural, syntactic and statistical pattern recognition
A2 - Yeung, DY
A2 - Kwok, JT
A2 - Fred, A
A2 - Roli, F
A2 - de Ridder, D
PB - Springer
CY - Berlin-Heidelberg
T2 - Joint IAPR International Workshops SSPR 2006 and SPR 2006, Hong Kong, China
Y2 - 17 August 2006 through 19 August 2006
ER -