Combining Binary Classifiers for a Multiclass Problem with Differential Privacy

Multiclass classification problem is often solved by combing binary classifiers into ensembles. While this is required for inherently binary classifiers, such as SVM, it also provides performance advantages for other classifiers. In this paper, we address the problem of combining binary classifiers into ensembles in the differentially private data publishing framework, where the data privacy is achieved by anonymization. The main idea of this paper is to counter the inevitable loss of data quality due to anonymization of the data by building an ensemble of binary classifiers, and then to use an error-correcting approach to obtain a class decision from this ensemble. We describe the proposed algorithm and present the results of extensive experimentation on synthetic and UC Irvine data. We find that while building ensembles after anonymization leads to no change in classifier accuracy, preparing the data for ensembles prior to anonymization improves accuracy in most of the cases.