Supervised Feature Discretization with a Dynamic Bit-Allocation Strategy

The use of feature discretization (FD) is useful in several machine learning and pattern recognition tasks. By attaining adequate and compact data representations, FD may improve the performance of many methods. It is often the case that learning with discrete data representations yields both lower training time and better accuracy, as compared to the use of the original features. Moreover, FD may also allow for a better human understanding/interpretation of the data. However, many FD techniques are sub-optimal, in the sense that they do not take into account feature interdependencies, as discretization is carried out. In this paper, we propose a dynamic supervised FD technique, addressing feature interdependencies. Our method selects the discretization cut-points by simultaneously maximizing two criteria: the dependency between the discretized features and the class label; the independence among these features. The method performs an incremental bit allocation scheme, using mutual information (MI) as the dependency/independence measure. Experimental results on low and medium-dimensional datasets show that the proposed method often achieves better accuracy than other well-known supervised FD approaches.