A new term-weighting scheme for text classification using the odds of positive and negative class probabilities
Abstract
Text classification is a core technique for text mining and information retrieval. It has been applied to many applications in many different research and industrial areas. Term weighting schemes have to assign an appropriate weight to each term to obtain a high text classification performance. Although term weighting is one of the important modules for text classification, and text classification has different peculiarities from those in information retrieval, many term weighting schemes used in information retrieval, such as tf.idf, have also been utilized in text classification in the same manner. The peculiarity of text classification that differs most from information retrieval is the existence of class information. Therefore, this paper proposes a new term weighting scheme that utilizes class information using positive and negative class distributions. As a result, the proposed scheme, log tf.TRR, consistently performs better than other schemes using class information, as well as traditional schemes such as tf.idf.