A Comparison of State-of-the-art Classification Techniques for Expert Automobile Insurance Claim Fraud Detection

Several state-of-the-art binary classification techniques are experimentally evaluated in the context of expert automobile insurance claim fraud detection. The predictive power of logistic regression, C4.5 decision tree, k-nearest neighbor, Bayesian learning multilayer perceptron neural network, least-squares support vector machine, naive Bayes, and tree-augmented naive Bayes classification is contrasted. For most of these algorithm types, we report on several operationalizations using alternative hyperparameter or design choices. We compare these in terms of mean percentage correctly classified (PCC) and mean area under the receiver operating characteristic (AUROC) curve using a stratified, blocked, ten-fold cross-validation experiment. We also contrast algorithm type performance visually by means of the convex hull of the receiver operating characteristic (ROC) curves associated with the alternative operationalizations per algorithm type. The study is based on a data set of 1,399 personal injury protection claims from 1993 accidents collected by the Automobile Insurers Bureau of Massachusetts. To stay as close to real-life operating conditions as possible, we consider only predictors that are known relatively early in the life of a claim. Furthermore, based on the qualification of each available claim by both a verbal expert assessment of suspicion of fraud and a ten-point-scale expert suspicion score, we can Stijn Viaene, Bart Baesens, and Guido Dedene are at the K. U. Leuven Department of Applied Economic Sciences, Leuven, Belgium. Richard Derrig is with the Automobile Insurers Bureau of Massachusetts, Boston. Presented at Fifth International Congress on Insurance: Mathematics & Economics July 23-25, 2001, Penn State University. This work was sponsored by the KBC Insurance Research Chair Management Informatics at the K. U. Leuven Department of Applied Economic Sciences. The KBC Research Chair was set up in September 1997 as a pioneering collaboration between the Leuven Institute for Research on Information Systems and the KBC Bank & Insurance group. We are grateful to the Automobile Insurers Bureau (AIB) of Massachusetts and the Insurance Fraud Bureau (IFB) of Massachusetts for providing us with the data that was used for this benchmark study.