Effective and Efficient Pruning of Meta-Classifiers in a Distributed Data Mining System

Distributed data mining systems aim to discover and combine useful information that is distributed across multiple databases. One of the main challenges is the design of effective and efficient methods to combine multiple models computed over multiple distributed sources that scale well over many large distributed databases. We describe in detail several methods that evaluate, prune and combine large collections of imported models computed at remote sites into efficient and scalable meta-classifiers. We demonstrate and evaluate the pruning methods by detailing many experiments performed on actual credit card data sets supplied by collaborating financial institutions, where the target learning task is fraud detection. We show that pruned meta-classifiers can sustain or even improve predictive performance at a substantially higher throughput, compared to the unpruned meta-classifiers.