BACKGROUND/AIMS Large clinical trials including patients with uncommon diseases involve assessors in different geographical locations, resulting in considerable inter-rater variability in assessment scores. As video recordings of examinations, which can be individually rated, may eliminate such variability, we measured the agreement between a single video rater and multiple examining physicians...

OBJECTIVE To discover how reliably speech and language therapists could diagnose apraxia of speech using their clinical judgement, by measuring whether they were consistent (intra-rater reliability), and whether their diagnoses agreed (inter-rater reliability). DESIGN Video clips of people with communication difficulties following stroke were rated by four speech and language therapists who w...

The study investigated the effects of three commonly employed rater training procedures on rating accuracy novice ESL essay raters. first procedure involved going through a set benchmarked scripts with scores, second assessing before viewing scores. third was combination former and latter. A pre, post delayed post-experimental research design employed. Data were collected before, immediately af...

The score reliability of language performance tests has attracted increasing interest. Classical Test Theory cannot examine multiple sources of measurement error. Generalizability theory extends Classical Test Theory to provide a practical framework to identify and estimate multiple factors contributing to the total variance of measurement. Generalizability theory by using analysis of variance ...

BACKGROUND Assessment centres are an accepted method of recruitment in industry and are gaining popularity within medicine. We describe the development and validation of a selection centre for recruitment to speciality training in anaesthesia based on an assessment centre model incorporating the rating of candidate's non-technical skills. METHODS Expert consensus identified non-technical skil...

BACKGROUND European and national administrative legislation require objective evaluation systems for organ lesions at pig slaughter. These results can be used as basis for herd health improvement programs by farmers and their consulting veterinarians. Various studies have shown that the current evaluation and recording of lesions by authorized meat inspectors are not reliable and produce signif...

BACKGROUND Although neighborhood characteristics have important relationships with health outcomes, direct observation involves imperfect measurement. The African American Health (AAH) study included two observer neighborhood rating systems (5-item Krause and 18-item AAH Neighborhood Assessment Scale [NAS]), initially fielded at two different waves. Good measurement characteristics were previou...