In this paper we review the Receiver Operating Characteristic ROC curve and the test statistic in relation to the analysis of a confusion matrix We then show how these two methods are related and propose an extension to the ROC curve so that it shows contours of values These contours can be used to provide further insight into the appropriate setting of the decision threshold for a particular a...

In receiver operating characteristic (ROC) analysis, the sampling variability can result in uncertainties of performance measures. Thus, while evaluating and comparing the performances of algorithms, the measurement uncertainties must be taken into account. The key issue is how to calculate the uncertainties of performance measures in ROC analysis. Our ultimate goal is to perform the significan...

Evaluating classifier performance with ROC curves is popular in the machine learning community. To date, the only method to assess confidence of ROC curves is to construct ROC bands. In the case of severe class imbalance, ROC bands become unreliable. We propose a generic framework for classifier evaluation to identify the confident segment of an ROC curve. Confidence is measured by Tango’s 95%-...

OBJECTIVE This was an exploratory analysis to develop a new way of representing BILAG-2004 system scores longitudinally that would be clinically meaningful and easier to analyse in comparison with multiple categorical variables. METHODS Data from a multicentre longitudinal study of SLE patients (the BILAG-2004 index and therapy collected at every visit) were used. External responsiveness anal...

It has been known for over 40 years that there are two fundamentally different kinds of detection tasks in the theory of signal detectability. The Type 1 task is to distinguish between events defined independently of the observer; the Type 2 task is to distinguish between one's own correct and incorrect decisions about those Type 1 events. For the Type 1 task, the behavior of the detector can b...

This paper demonstrates that, for large-scale tests, the match and non-match similarity scores have no specific underlying distribution function. The forms of these distribution functions require a nonparametric approach for the analysis of the fingerprint similarity scores. In this paper, we present an analysis of the discrete distribution functions of the match and non-match similarity scores...

Developing novel nuclear cardiology approaches requires evaluating their diagnostic ability. Statistical measures such as sensitivity, specificity, receiver operating characteristic (ROC) curve, and area under the curve (AUC) are useful to evaluate the diagnostic value of novel imaging parameters. This paper reviews key statistical methods used in the evaluation of diagnostic tests and highligh...

Receiver operating characteristic (ROC) curves are widely used to measure the discriminating power of medical tests and other classification procedures. In many practical applications, the performance of these procedures can depend on covariates such as age, naturally leading to a collection of curves associated with different covariate levels. This paper develops a Bayesian heteroscedastic sem...

Dual-process models of the word-frequency mirror effect posit that low-frequency words are recollected more often than high-frequency words, producing the hit rate differences in the word-frequency effect, whereas high-frequency words are more familiar, producing the false-alarm-rate differences. In this pair of experiments, the authors demonstrate that the analysis of receiver operating charac...