This paper compares the robustness of information retrieval (IR) metrics to incomplete relevance assessments, using four different sets of graded-relevance test collections with submitted runs – two from TREC and two from NTCIR. We investigate the effect of reducing the original relevance data on discriminative power (i.e., how often statistical significance can be detected given the probabilit...

This paper presents the evaluation of the translation quality and Cross-Lingual Information Retrieval (CLIR) performance when using session information as the context of queries. The hypothesis is that previous queries provide context that helps to solve ambiguous translations in the current query. We tested several strategies on the TREC 2010 Session track dataset, which includes query reformu...

The accuracy of a collaborative-filtering system largely depends on two factors: the quality of the recommendation algorithm and the number and quality of the available product ratings. In general, the more ratings are elicited from the users, the more effective the recommendations are. However, not all the ratings are equally useful and specific techniques, which are defined as rating elicitat...

This paper describes experiments carried out at NiCT for the TREC 2009 Entity Ranking track. Our main study is to develop an effective approach to rank entities via measuring the “similarities” between supporting snippets of entities and input query. Three models are implemented to this end. 1) The DLM regards entity ranking as a task of calculating the probabilities of generating input query g...

An interesting issue “whether the factual search result is better than the satisfactory one?” is discussed in this paper, and all effort is to illustrate the publicly recognized conception “the supreme satisfactory result is optimal” could lead Information Retrieval astray. By contrast, we propose a new hypothesis “the factually result should be the optimal although sometimes unsatisfactory”. T...

― 107 ― KECIR An Information Retrieval System for IR4QA Task Dongfeng Cai, Shengqiao Kang, Yu Bai, Peiyan Wang Knowledge Engineering Research Center, Shenyang Institute of Aeronautical Engineering [email protected] Abstract This paper describes our work on the subtask of simplified Chinese monolingual information retrieval for question answering system at ntcir-8. We use the lemur toolkit to...

An interesting issue “whether the factual search result is better than the satisfactory one?” is discussed in this paper, and all effort is to illustrate the publicly recognized conception “the supreme satisfactory result is optimal” could lead Information Retrieval astray. By contrast, we propose a new hypothesis “the factually result should be the optimal although sometimes unsatisfactory”. T...