Abstract Receiver operating characteristic (ROC) curves are used ubiquitously to evaluate scores, features, covariates or markers as potential predictors in binary problems. We characterize ROC from a probabilistic perspective and establish an equivalence between cumulative distribution functions (CDFs). These results support subtle shift of paradigms the statistical modelling curves, which we ...

We propose serial correlation robust asymptotic confidence bands for the receiver operating characteristic (ROC) curves estimated by quasi-maximum likelihood in the binormal model. Our simulation experiments confirm that this new method performs fairly well in finite samples. The conventional procedure is found to be markedly undersized in terms of yielding empirical coverage probabilities lowe...

In this paper we review the Receiver Operating Characteristic ROC curve and the test statistic in relation to the analysis of a confusion matrix We then show how these two methods are related and propose an extension to the ROC curve so that it shows contours of values These contours can be used to provide further insight into the appropriate setting of the decision threshold for a particular a...

In receiver operating characteristic (ROC) analysis, the sampling variability can result in uncertainties of performance measures. Thus, while evaluating and comparing the performances of algorithms, the measurement uncertainties must be taken into account. The key issue is how to calculate the uncertainties of performance measures in ROC analysis. Our ultimate goal is to perform the significan...

Evaluating classifier performance with ROC curves is popular in the machine learning community. To date, the only method to assess confidence of ROC curves is to construct ROC bands. In the case of severe class imbalance, ROC bands become unreliable. We propose a generic framework for classifier evaluation to identify the confident segment of an ROC curve. Confidence is measured by Tango’s 95%-...

It has been known for over 40 years that there are two fundamentally different kinds of detection tasks in the theory of signal detectability. The Type 1 task is to distinguish between events defined independently of the observer; the Type 2 task is to distinguish between one's own correct and incorrect decisions about those Type 1 events. For the Type 1 task, the behavior of the detector can b...

There are different approaches in the literature for the assessment of steganographic algorithms and steganalytic attacks. In the early papers it was considered sufficient to show the existence of an effect for one or a few examples only. The more the area of steganography evolved, the more diverse became the goals and the harder to measure the improvements. Many branches of science are facing ...

UNLABELLED Receiver operating characteristic (ROC) analysis is usually applied in bioinformatics to evaluate the abilities of biological markers to differentiate between the presence or absence of a disease. It includes the derivation of the useful scalar performance measure area under the ROC curve for binary classification tasks. As real applications often deal with more than two classes, mul...