University of Hagen at CLEF 2005: Towards a Better Baseline for NLP Methods in Domain-Specific Information Retrieval

The third participation of the University of Hagen at the German Indexing and Retrieval Test (GIRT) task of the Cross Language Evaluation Campaign (CLEF 2005) aims at providing a better baseline for experiments with natural language processing (NLP) methods in domainspecific information retrieval (IR). Our monolingual experiments with the German document collection are based on a setup combining several methods to achieve a better performance. The setup includes an entry vocabulary module (EVM), query expansion with semantically related concepts, and a blind feedback technique. The monolingual experiments focus on comparing two techniques for constructing database queries: creating a ’bag of words’ and creating a semantic network by means of a syntactico-semantic parser for a deep linguistic analysis of the query. The best performance in the official experiments was achieved by a setup using staged logistic regression, a query expansion with semantically related concepts, an entry vocabulary module, a deep linguistic analysis of the query, and blind feedback (0.2875 mean average precision (MAP)). Additional experiments showed a performance improvement when changing to the basic Okapi BM25 search (0.3878 MAP). For the bilingual experiments, the English topics are translated into German queries with several machine translation services available online (Systran, Free translation, WorldLingo, and Promt). Each set of translated topics is processed separately with the same techniques as in the monolingual experiments. The best performance was achieved with a query translation by Promt with a simple keyword extraction from the translation (0.2399 MAP with a staged logistic regression approach vs. 0.2807 MAP with Okapi BM25).