'Objective' evaluation means evaluation using numbers (metrics) which can be calculated without intervention by humans. An obvious advantage is that human eeort is reduced. It is also claimed that it is less biased by human opinion, but in this text it is shown that there are a lot of choices to be made when applying these methods, which introduces bias at a diierent level. Two objective evalua...

New technologies have led to an “explosion” of data available to document states and processes in very many fields. Tools of data mining are being used to extract relevant information. If this information is used in decision making, analytical statistics can provide formal tests comparing the outcomes of different scenarios. Statistics has traditionally dealt with limited information, both in t...

The provenance, or lineage, of a workflow data product can be reconstructed by keeping a complete trace of one workflow execution. This lineage information, however, is likely to be both imprecise, because of the black-box nature of the services that compose the workflow, and noisy, because of the many trivial data transformations that obscure the intended purpose of the workflow. In this paper...

This paper presents an approach to correct chromatic distortion within an image (vignetting) and to compensate for color response differences among similar cameras which equip a team of robots, based on Evolutionary Algorithms. Our black-box approach does not make assumptions concerning the physical/geometrical roots of the distortion, and the efficient implementation is suitable for real time ...

Beim Testen kommt der Wahl der Testfälle eine entscheidende Bedeutung zu, denn mit der Festlegung der Testfälle wird über die Chancen zur Fehlerentdeckung entschieden. Viele Untersuchungen gehen der Frage nach, ob beim Black-Box-Test oder beim Glass-Box-Test effektivere Testfälle entstehen. Heute ist sich die Literatur weitgehend einig, dass die beiden Testverfahren keine Alternativen bilden, s...

Many practical recommendation systems have been studied, and also the services based on such recommendation systems have been opened in real world. The main research questions of this work are i) how these recommendation services provide users with useful information, and ii) how different the results from the systems are from each other. In this paper, we propose a black-box evaluation framewo...

Leakage resilience and black-box impossibility results in cryptography Ali Juma Doctor of Philosophy Graduate Department of Computer Science University of Toronto 2011 In this thesis, we present constructions of leakage-resilient cryptographic primitives, and we give black-box impossibility results for certain classes of constructions of pseudo-random

This paper presents a methodology for automating the evaluation of partial designs using black-box testing techniques. The methodology generates black-box evaluation testsets using a novel semantic graph data model which maintains the relationships between design and requirements data. The testset is used to simulate the design module and the related requirements, thus generating a set of expec...

Bringing transparency to black-box decision making systems (DMS) has been a topic of increasing research interest in recent years. Traditional active and passive approaches to make these systems transparent are often limited by scalability and/or feasibility issues. In this paper, we propose a new notion of black-box DMS transparency, named, temporal transparency, whose goal is to detect if/whe...

Testing a black-box system without recourse to a specification is difficult, because there is no basis for estimating how many tests will be required, or to assess how complete a given test set is. Several researchers have noted that there is a duality between these testing problems and the problem of inductive inference (learning a model of a hidden system from a given set of examples). It is ...