An Empirical Comparison of Hierarchical vs. Two-Level Approaches to Multiclass Problems

The ECOC framework provides a powerful and popular method for solving multiclass problems using a multitude of binary classifiers. We had recently introduced the Binary Hierarchical Classifier (BHC) architecture that addresses multiclass classification problems using a set of binary classifiers arranged as a tree. Unlike ECOCs, the BHC groups classes according to their natural affinities in order to make each binary problem easier. However it cannot exploit the powerful error correcting properties of an ECOC ensemble that can provide good results even when individual classifiers are weak. Using welltuned SVMs as the base classifiers, we provide a comparison of these two diverse approaches using a variety of datasets. The results show that while there is no clear advantage to either technique in terms of classification accuracy, BHCs typically achieve this performance using fewer classifiers, and have the added advantage of automatically generating a hierarchy of classes. Such hierarchies often provide a valuable tool for extracting domain knowledge, and achieve better results when coarser granularity of the output space is acceptable.