Constructing Time Decompositions for Analyzing Time-Stamped Documents

Extraction of sequences of events from news and other documents based on the publication times of these documents has been shown to be extremely effective in tracking past events. This paper addresses the issue of constructing an optimal decomposition of the time period associated with a given document set, i.e., a decomposition with the smallest number of subintervals, subject to no or limited loss of information. We introduce the notion of the compressed interval decomposition, where each subinterval consists of consecutive time points having identical information content. We define optimality, and show that any optimal information preserving decomposition of the time period is a refinement of the compressed interval decomposition. We define several special classes of measure functions (functions that compute the significant information from document sets), based on their effect on the information computed as document sets are combined. These classes are used in developing algorithms for computing an optimal information preserving decomposition of the time period of a given document set. We also define the notion of information loss of a time decomposition of a given document set and give an efficient algorithm for computing an optimal lossy decomposition. We discuss the effectiveness of our algorithms on the Reuters–21578, Distribution 1.0 data set and a subset of Medline abstracts.