Fault masking can reduce the effectiveness of a test suite. We propose an information theoretic measure, Squeeziness, as the theoretical basis for avoiding fault masking. We begin by explaining fault masking and the relationship between collisions and fault masking. We then define Squeeziness and demonstrate by experiment that there is a strong correlation between Squeeziness and the likelihood...

In this paper we examine a procedure for the analysis of data produced by the dynamic profiling of Java programs. In particular, we describe the issues involved in dynamic analysis, propose a metric for discrimination between the resulting data sets, and examine its application over different test suites and compilers.

Regression testing is an expensive testing procedure utilized to validate modified software. Regression test selection techniques attempt to reduce the cost of regression testing by selecting a subset of a program’s existing test suite. Safe regression test selection techniques select subsets that, under certain well-defined conditions, exclude no tests (from the original test suite) that if ex...

We investigate a range of crosslingual web retrieval tasks using the test suite of the CLEF 2005 WebCLEF track, which features a stream of known-item topics in various languages. Our main findings are: (i) straightforward indexing and retrieval is effective for mixed monolingual web retrieval; (ii) standard machine translation methods are effective for bilingual web retrieval; but (iii) standar...

Parsers have evolved significantly in the last decades, but currently big and accurate improvements are needed to enhance their performance. ParTes, a test suite in Spanish and Catalan for parsing evaluation, aims to contribute to this situation by pointing to the main factors that can decisively improve the parser performance.

In this paper we implement GAs that have one or more parameters that are adjusted during the run. In particular we use an existing self-adaptive mutation rate mechanism, propose a new mechanism for self-adaptive crossover rates, and redesign an existing variable population size model. We compare the simple GA with GAs featuring only one of the parameter adjusting mechanisms and with a GA that a...

test suite System specification Test refinement System design Set of tests covering 100% of states Implementation

We present PROTEST, a test suite for the evaluation of pronoun translation by MT systems. The test suite comprises 250 hand-selected pronoun tokens and an automatic evaluation method which compares the translations of pronouns in MT output with those in the reference translation. Pronoun translations that do not match the reference are referred for manual evaluation. PROTEST is designed to supp...

This paper presents results from a head-to-head comparison of two Lagrangian methods for solutions to the two-dimensional, incompressible convection-diffusion equations. The first Lagrangian method is an axisymmetric core spreading method using Gaussian basis functions. The second method uses deforming elliptical Gaussian basis functions. Previous results show that the first method has second-o...