Validation of Two-Sample Bootstrap in ROC Analysis on Large Datasets Using AURC

Sampling variability results in uncertainties of measures. The nonparametric twosample bootstrap method has been used to compute uncertainties of measures in receiver operating characteristic (ROC) analysis on large datasets, such as the standard error (SE) of the equal error rate in biometrics, the SE of a detection cost function in speaker recognition evaluation, and others. Specifically, the SE of the area under ROC curve (AURC) can be computed analytically using the Mann-Whitney statistic. It can also be calculated using the nonparametric two-sample bootstrap method. The analytical result could be treated as a ground truth. The relative errors of bootstrap-method results with respect to the analytical-method results using different matching algorithms were examined, and they were quite small. Hence, this validates the nonparametric two-sample bootstrap method applied in ROC analysis on large datasets. Index Terms -ROC analysis, bootstrap, area under ROC curve, uncertainty, standard error, biometrics, speaker recognition. * Tel: + 301-975-6996; fax: + 301-975-5287. E-mail address: [email protected].