This study exploits statistical redundancy inherent in natural language to automatically predict scores for essays. We use a hybrid feature identification method, including syntactic structure analysis, rhetorical structure analysis, and topical analysis, to score essay responses from test-takers of the Graduate Management Admissions Test (GMAT) and the Test of Written English (TWE). For each e...

INTRODUCTION Quality assessment of included studies is an important component of systematic reviews. OBJECTIVE The authors investigated inter-rater and test-retest reliability for quality assessments conducted by inexperienced student raters. DESIGN Student raters received a training session on quality assessment using the Jadad Scale for randomised controlled trials and the Newcastle-Ottaw...

OBJECTIVE Chronic pain from rheumatic and musculoskeletal conditions is associated with cortical changes and altered tactile acuity. Tactile acuity is considered a clinical signature of primary somatosensory representation. The two-point discrimination (TPD) threshold is increasingly used both clinically and in research. Remarkably, the reliability and precision of the measure at commonly used ...

Measuring the entorhinal cortex (ERC) is challenging due to lateral border discrimination from the perirhinal cortex. From a sample of 39 nondemented older adults who completed volumetric image scans and verbal memory indices, we examined reliability and validity concerns for three ERC protocols with different lateral boundary guidelines (i.e., Goncharova, Dickerson, Stoub, & deToledo-Morrell, ...

Dental and dental hygiene faculty members often do not provide consistent instruction in the clinical environment, especially in tasks requiring clinical judgment. From previous efforts to calibrate faculty members in calculus detection using typodonts, researchers have suggested using human subjects and emerging technology to improve consistency in clinical instruction. The purpose of this pil...

The interrater reliability of the Supports Intensity Scale (SIS) was investigated under the condition that interviewers had to have been trained and/or experienced in its administration and scoring. Both corrected and noncorrected Pearson's product-moment coefficients were generated to assess interinterviewer, interrespondent, and mixed interrater reliability. The correlation coefficients for t...

The six competencies of the Accreditation Council for Graduate Medical Education include the lifelong learning skills of evidence-based medicine (EBM)/information mastery. We developed and tested an Objective Structured Clinical Examination (OSCE) station that would measure these skills in family medicine residents. This EBM OSCE station is a 30-minute station within a regular OSCE exam. It use...

A quasi-experiment was conducted to investigate the effects of frame-of-reference training on the quality of competency modeling ratings made by consultants. Human resources consultants from a large consulting firm were randomly assigned to either a training or a control condition. The discriminant validity, interrater reliability, and accuracy of the competency ratings were significantly highe...

objective: to examine the inter-rater reliability of triages performed by the electronic triage system (ets) which has recently developed and used in hospital emergency department (ed). methods: this cross-sectional study was conducted prospectively and studied 408 visitors of tabriz imam reza hospital’s ed. the variables of interest were age, sex, nurse-assigned triage category, physician-assi...

BACKGROUND AND PURPOSE The modified Rankin Scale (mRS) is widely used to assess global outcome after stroke. The aim of the study was to examine rater variability in assessing functional outcomes using the conventional mRS, and to investigate whether use of a structured interview (mRS-SI) reduced this variability. METHODS Inter-rater agreement was studied among raters from 3 stroke centers. F...