An important assumption underlying meaningful comparisons of scores in rater-mediated assessments is that measurement is commensurate across raters. When raters differentially apply the standards established by an instrument, scores from different raters are on fundamentally different scales and no longer preserve a common meaning and basis for comparison. In this study, we developed a method t...

GOAL To assess the reliability of a reference standard for an information extraction task. SETTING Twenty-four physician raters from two sites and two specialties judged whether clinical conditions were present based on reading chest radiograph reports. METHODS Variance components, generalizability (reliability) coefficients, and the number of expert raters needed to generate a reliable ref...

Background: Clock drawing is part of the Montreal Cognitive Assessment (MoCA) test but may have administration and scoring limitations. We assessed (1) the reliability of the MoCA clock criteria relative to a published error scoring approach, (2) whether command-only administration could distinguish dementia from cognitively intact individuals and (3) the value of adding a clock copy condition ...

The chair-side work posture of dental hygienists has long been a concern because of health-related problems potentially caused or exacerbated by poor posture. The purpose of this study was to investigate if using magnification loupes improved dental hygiene students' posture during provision of treatment. The treatment chosen was hand-scaling, and the effect of the timing of introduction of the...

Abstract Summary writing is essential for academic success, and has attracted renewed interest in research large-scale language test. However, less attention been paid to the development evaluation of scoring scales summary writing. This study reports on validation a rubric that represented an approach scale with limited resources out consideration practicality. Participants were 83 students th...

BACKGROUND Traditional focus groups have been essential to facial aesthetics research. Although they are currently the criterion standard in acquiring facial attractiveness ratings, they retain many shortcomings. This study's objectives were twofold: to determine whether attractiveness scores obtained from a social network site correlate with those from the traditional focus group method; and t...

Raters of corporations play an important role in assessing domains ranging from sustainability to corporate governance to best workplaces. Scholars increasingly rely on these ratings to test theories about corporate social responsibility (CSR), corporate governance and the influence of stakeholders. Though these raters frequently develop sophisticated methodologies, we find they often diverge i...

The evolution of cooperation requires some mechanism that reduces the risk of exploitation for cooperative individuals. Recent studies have shown that men with wide faces are anti-social, and they are perceived that way by others. This suggests that people could use facial width to identify anti-social men and thus limit the risk of exploitation. To see if people can make accurate inferences li...

With the increase of visualization platforms targeting novices, researchers are now focusing on gathering insights regarding novice user practices. We describe the design and evaluation of Exploration Views (EV), a system that allows novice visualization users to easily build and customize Business Intelligence information dashboards. EV provides an intuitive environment for dynamically creatin...

With the increase of visualization platforms targeting novices, researchers are now focusing on gathering insights regarding novice user practices. We describe the design and evaluation of Exploration Views (EV), a system that allows novice visualization users to easily build and customize Business Intelligence information dashboards. EV provides an intuitive environment for dynamically creatin...