System development efforts depend to a large degree upon how well information systems (IS) managers, IS specialists, and IS users work together in a project team structure. Yet, these individuals frequently work under different perceptions about matters of importance to development, management, and success. This paper introduces a framework for examining IS specialists’ skill requirements from ...

This paper presents the techniques implemented in GISTEXTER for producing extracts and abstracts from both single and multiple documents. These techniques promote the belief that highly coherent summaries may be generated when using textual information identified by the Information Extraction technology. The results of GISTEXTER in the DUC-2002 evaluations account for the advantages of using th...

In this paper, we investigate information validation tasks that are initiated as queries from either automated agents or humans. We introduce OpenEval, a new online information validation technique, which uses information on the web to automatically evaluate the truth of queries that are stated as multiargument predicate instances (e.g., DrugHasSideEffect(Aspirin, GI Bleeding))). OpenEval gets ...

This paper studies a principal–agent relation in which the principal’s private information about the agent’s effort choice is more accurate than a noisy public performance measure. For some contingencies the optimal contract has to specify ex post inefficiencies in the form of inefficient termination (firing the agent) or third–party payments (money burning). We show that money burning is the l...

Information retrieval (IR) research has primarily consisted of two paradigms: systemsoriented research and user studies. Systems-oriented research includes IR algorithm development and evaluation and, to some degree, human-system interaction. User studies include human information behavior and information seeking research. These paradigms have contributed new IR algorithms and insights into IR ...

One of the challenges of modern information retrieval is to adequately evaluate Information Retrieval System (IRS) in order to estimate future performance in a specified application domain. Since there are many algorithms in literature the decision to select one for usage depends mostly on the evaluation of the systems’ performance in the domain. This paper presents how visual and scalar evalua...

This paper presents Russian information retrieval evaluation initiative and results obtained during first year. In particular, we describe first ROMIP seminar, used Cyrillic Web collection and search tasks as well as ongoing efforts on ROMIP’2004.

Online evaluation is one of the most common approaches to measure the effectiveness of an information retrieval system. It involves fielding the information retrieval system to real users, and observing these users’ interactions in-situ while they engage with the system. This allows actual users with real world information needs to play an important part in assessing retrieval quality. As such,...