Our detailed investigation into various approaches to evaluate Information Systems Development methods has shown that numerous attempts to assess these methods only yield inconclusive and questionable results. There are two general trends as to how various criteria for evaluation are organised. There are relatively ad hoc lists of criteria for evaluation, and systematically organised frameworks...

Recently, speech recognition performance has been drastically improved by statistical methods and huge speech databases. Now performance improvement under such realistic environments as noisy conditions is being focused on. Since October 2001, we from the working group of the Information Processing Society in Japan have been working on evaluation methodologies and frameworks for Japanese noisy ...

The extraction of relations between named entities from natural language text is a longstanding challenge in information extraction, especially in large-scale. A major challenge for the advancement of this research field has been the lack of meaningful evaluation frameworks based on realistic-sized corpora. In this paper we propose a framework for large-scale evaluation of relation extraction s...

Usability Evaluation (UE) is defined as the process of measuring usability and recognizing explicit usability problems. UE in mobile application aims to identify the main issues in the user interface that may lead to human error, terminate the user interaction with the mobile application and cause user frustration. UE frameworks are considered as major success factor and led to higher acceptanc...

This paper proposes a comprehensive and multi-dimensional feature analysis framework for evaluating and comparing methodologies for developing multi-agent systems (MAS). Developed from a synthesis of various existing evaluation frameworks, the novelty of our framework lies in the high degree of its completeness and the relevance of its evaluation criteria. The paper also presents a pioneering e...

In addressing enterprise integration problems, a diversity of technologies such as CORBA and XML were promoted, yet no single integration technology solves all integration problems. As a result, a new generation of software called Enterprise Application Integration (EAI) is emerging to addresses many integration problems by combining a diversity of integration technologies (e.g. message brokers...

The paper tackled the issue of arguments evaluation in weighted bipolar argumentation frameworks (i.e., graphs whose arguments have basic strengths, and may be both supported and attacked). We introduce axioms that an evaluation method (or semantics) could satisfy. Such axioms are very useful for judging and comparing semantics. We then analyze existing semantics on the basis of our axioms, and...

A topic tracking system that combines elements from vector space and language modeling frameworks to compute document scores is described. The model is used for both the traditional TDT topic tracking evaluation design and the new supervised adaptive topic tracking evaluation. Results indicate that supervised adaptation and score normalization should be more closely coupled, and that current te...

nowadays a growing number of implementing informationtechnology service management (itsm) frameworks proves the trends toimprove the it services and processes in organizations. in order toimplement an itsm projects successfully in an organization, it isessential to select a suitable itsm solution. evaluation and selection ofthe itsm software packages is complicated and time consuming decisionma...

in a recent editorial for this journal, el-jardali and fadlallah proposed a new framework for knowledge translation (kt) in healthcare. many such frameworks already exist; thus, new entrants to the field must be scrutinized in regard to their unique contributions to advancing understanding and practice. the el-jardali and fadlallah framework focuses on policy-level discussions, a relatively und...