Analysis of Statistical Keyword Extraction Methods for Incremental Clustering

Incremental clustering is a very useful approach to organize dynamic text collections. Due to the time/space restrictions for incremental clustering, the textual documents must be preprocessed to maintain only their most important information. Statistical keyword extraction methods from single documents are useful in this scenario. However, different statistical methods have different assumptions about the properties of keywords in a text, and different methods extract different set of keywords. In this paper we analyze the different methods for keyword extraction and the impact of the number of keywords on the quality of the incremental clustering. We also define a framework for statistical keyword extraction.