Benchmarking and peer review are essential elements of the so-called EU open method of coordination (OMC) which has been contested in the literature for lack of effectiveness. In this paper we compare benchmarking and peer review procedures as used by the EU with those used by the OECD. Different types of benchmarking and peer review are distinguished and pitfalls for (international) benchmarki...

Benchmarking of spatio-temporal databases is an issue of growing importance. In case large real data sets are not available, benchmarking requires the generation of artificial data sets following the realworld behavior of spatial objects that change their locations, shapes and sizes over time. In this report, a generator for changing raster data, called “Generator of Time-Evolving Regional Data...

Benefits are indissolubly associated to cost, as specific cost levels will always raise questions regarding the accompanying benefit levels. Contrary to cost benchmarking, the benchmarking of benefits, and especially the benefit constituents which should be measured, seem hardly developed. This study presents the IT benchmarking of housing associations over a period of seven years. Although cos...

Best practice benchmarking is now relatively common in all sectors of the economy, including the voluntary and public sectors. It is even being adopted by small and medium sized enterprises, although seldom by very small firms. The gains in terms of improved performance of key processes are often assumed to be self-evident, yet systematic evaluation of the costs and benefits of benchmarking is ...

This demonstration paper illustrates RMASBench, a new benchmarking system based on the RoboCup Rescue Agent simulator. The aim of the system is to facilitate benchmarking of coordination approaches in controlled settings for dynamic rescue scenarios. In particular, the key features of the systems are: i) programming interfaces to plug-in coordination algorithms without the need for implementing...

The incentive regulation of costs related to physical and cyber security in electricity networks is an important but relatively unexplored and ambiguous issue. These costs can be part of a cost efficiency benchmarking or alternatively dealt separately. This paper discusses the issues and proposes on the options for incorporating network security costs within incentive regulation in a benchmarki...

We present an any-time performance assessment for benchmarking numerical optimization algorithms in a black-box scenario, applied within the COCO benchmarking platform. The performance assessment is based on runtimes measured in number of objective function evaluations to reach one or several quality indicator target values. We argue that runtime is the only available measure with a generic, me...

Using a combination of metamodels, ontologies, green performance indicators and metrics, we apply a novel approach in Semantic Business Process Benchmarking to the area of Green Business Process Management (Green BPM). Up to now, process benchmarking has mainly been a manual process; the approach described and empirically evaluated in this paper partially automates the time-consuming and costly...