This paper proposes an evaluation scheme to measure the performance of a system that detects hierarchical event structure for event coreference resolution. We show that each system output is represented as a forest of unordered trees, and introduce the notion of conceptual event hierarchy to simplify the evaluation process. We enumerate the desiderata for a similarity metric to measure the syst...

The NLToolset has been used to build a variety of information extraction applications, ranging from military message traffic to newswire accounts of corporate activity. AATM7 is an acronym for As Applied To MUC-7. AATM7 was not tailored specifically for MUC-7, but rather represents the NLToolset in a state of flux, as TIPSTER experimentation and the delivery of a real-world application were tak...

The session on Natural Language Evaluation focused on methods for evaluating text understanding systems. Beginning with the first Message Understanding Conference (MUCK-l) in 1987, there has been increasing focus on how to measure and evaluate text understanding systems. The MUCK-1 conference required developers to port their system to a common domain of Navy intelligence messages; MUCK-2 (May ...

The brain representations of words and their referent actions and objects appear to be strongly coupled neuronal assemblies distributed over several cortical areas. In this work we describe the implementation of a cell assembly-based model of several visual, language, planning, and motor areas to enable a robot to understand and react to simple spoken commands. The essential idea is that differ...

Traditional approaches to the problem of extracting data from texts have emphasized handcrafted linguisti c knowledge. In contrast, BBN's PLUM system (Probabilistic Language Understanding Model) was developed as part of a DARPA-funded research effort on integrating probabilistic language models with more traditiona l linguistic techniques . Our research and development goals are • more rapid de...

Interpreting fttUy natural speech is an important goal for spoken language understanding systems. However, while corpus studies have shown that about 10% of spontaneous utterances contain selfcorrections, or REPAIRS, little is known about the extent to which cues in the speech signal may facilitate repair processing. We identify several cues based on acoustic and prosodic analysis of repairs in...

Certain content of a linguistic construction can project when the construction is embedded in entailment-canceling environments. For example, the conclusion that John smoked in the past from the utterance John stopped smoking still holds for John didn’t stop smoking, in which the original utterance is embedded under negation. There are two main approaches to account for projection phenomena. Th...

The sixth in a series of "Message Understanding Conferences", which are designed to promote and evaluate research in information extraction, was held last fall. MUC-6 introduced several innovations over prior MUCs, most notably in the range of different tasks for which evaluations were conducted. We describe the development of the "message understanding" task over the course of the prior MUCs, ...

T h e aim o f t h is disse rt at io n is t o ide nt ify h o w t h e spe c t rum c urre nt l y al l o c at e d fo r T V bro adc ast ing c an be re use d by so me o t h e r c o mmunic at io n syst e ms. T h e spe c t rum re use is al l o w e d o nl y if t h e c urre nt T V use rs are no t dist urbe d. T h e st udy addre sse s t h e pro bl e ms o f h o w t h e se c o ndary ac c e ss t o t h e spe ...