background and objectives: mthe emergency severity index (esi) triage is commonly used in clinical settings to determine the patients’ emergency severity. however, the reliability of this index is not sufficiently explored. the present study examines the inter-rater reliability of esi by comparing triage ratings as performed by nurses and physicians. methods: this prospective cross-sectional st...

The e-rater system 1 is an operational automated essay scoring system, developed at Educational Testing Service (ETS). The average agreement between human readers, and between independent human readers and e-rater is approximately 92%. There is much interest in the larger writing community in examining the system’s performance on nonnative speaker essays. This paper focuses on results of a stud...

BACKGROUND AND PURPOSE The modified Rankin Scale (mRS) is widely used to assess global outcome after stroke. The aim of the study was to examine rater variability in assessing functional outcomes using the conventional mRS, and to investigate whether use of a structured interview (mRS-SI) reduced this variability. METHODS Inter-rater agreement was studied among raters from 3 stroke centers. F...

OBJECTIVES The purpose of this study was to assess the inter-rater reliability and intra-rater reliability of medical record review for the detection of hospital adverse events. METHODS We conducted two stages retrospective medical records review of a random sample of 96 patients from one acute-care general hospital. The first stage was an explicit patient record review by two nurses to detec...

Background Quantifying the strength of the intrinsic foot muscles has been a challenge for clinicians and researchers. The reliable measurement of this strength is important in order to assess weakness, which may contribute to a variety of functional issues in the foot and lower leg, including plantar fasciitis and hallux valgus. This study reports 3 novel methods for measuring foot strength - ...

INTRODUCTION There is a need for reliable and practical interprofessional simulations that measure collaborative practice in outpatient/community scenarios where most health care takes place. The authors applied generalizability theory to examine reliability in an ambulatory care scenario using the following 2 trained observer groups: standardized patient (SP, actor) raters and those who receiv...

Objectives. To measure inter-rater agreement of overall clinical appearance of febrile children aged less than 24 months and to compare methods for doing so. Study Design and Setting. We performed an observational study of inter-rater reliability of the assessment of febrile children in a county hospital emergency department serving a mixed urban and rural population. Two emergency medicine hea...

Performance appraisals have been frequently investigated for inaccuracies attributable to raters. In addition to rater contributions, problems in measurement can arise from properties of instrumentation. If administered items do not parallel the full range of employee performance, a restriction of range can occur. Here employees are rated similarly (no distinction made) not because of rater err...

This article presents a latent distribution model for the analysis of agreement on dichotomous or ordered category ratings. The model includes parameters that characterize bias, category definitions, and measurement error for each rater or test. Parameter estimates can be used to evaluate rater performance and to improve classification or measurement with use of multiple ratings. A simple maxim...