Ordinal regression learning has characteristics of both multi-class classification and metric regression because labels take ordered, discrete values. In applications of ordinal regression, the misclassification cost among the classes often differs and with different misclassification costs the common performance measures are not appropriate. Therefore we extend ROC analysis principles to ordin...

Classification accuracy is the ability of a marker or diagnostic test to discriminate between two groups of individuals, cases and controls, and is commonly summarized using the receiver operating characteristic (ROC) curve. In studies of classification accuracy, there are often covariates that should be incorporated into the ROC analysis. We describe three different ways of using covariate inf...

The performance of a classifier can be improved by abstaining on uncertain instance classifications. The transformation from the original Receiver Operator Characteristic (ROC) curve to the curve obtained by abstention is provided. We include proofs on dominance of this new ROC curve to aid classifier selection and to show the effectiveness of the approach. For specific cost and class distribut...

Receiver operating characteristics (ROC) graphs are useful for organizing classifiers and visualizing their performance. ROC graphs are commonly used in medical decision making, and in recent years have been used increasingly in machine learning and data mining research. Although ROC graphs are apparently simple, there are some common misconceptions and pitfalls when using them in practice. The...

We address the problem of comparing the performance of classifiers. In this paper we study techniques for generating and evaluating confidence bands on ROC curves. Historically this has been done using one-dimensional confidence intervals by freezing one variable—the false-positive rate, or threshold on the classification scoring function. We adapt two prior methods and introduce a new radial s...

In this paper we review the Receiver Operating Characteristic ROC curve and the test statistic in relation to the analysis of a confusion matrix We then show how these two methods are related and propose an extension to the ROC curve so that it shows contours of values These contours can be used to provide further insight into the appropriate setting of the decision threshold for a particular a...

Receiver operating characteristic (ROC) curves are useful statistical tools for medical diagnostic testing. It has been proved its capability to assess diagnostic marker’s ability to distinguish between healthy and diseased subjects and to compare different diagnostic markers. In this paper we introduce non parametric ROC summary statistics to assess a ROC curve across the entire range of FPFs ...

Declaration This dissertation is submitted to the University of Bristol in accordance with the requirements of the degree of Bachelor of Science in the Faculty of Engineering. It has not been submitted for any other degree or diploma of any examining body. Except where specifically acknowledged, it is all the work of the Author. 3 ABSTRACT Machine Learning applications require learning algorith...