Generalizability theory provides a comprehensive framework for determining how multiple sources of measurement error affect scores from psychological assessments and using that information to improve those assessments. Although generalizability designs have traditionally been analyzed analyses variance (ANOVA) procedures, the same can be replicated extended structural equation models. We collec...

The assessment of creative problem solving (CPS) is challenging. Elements an procedure, such as the tasks that are used and raters who assess those tasks, introduce variation in student scores do not necessarily reflect actual differences students’ abilities. When creativity researchers evaluate procedures, they often inspect these elements separately. We show use Generalizability Theory allows...

The multiple mini-interview (MMI) has become an increasingly popular admissions method for selecting prospective students into professional programs (e.g., medical school). The MMI uses a series of short, labour intensive simulation stations and scenario interviews to more effectively assess applicants' non-cognitive qualities such as empathy, critical thinking, integrity, and communication. MM...

Direct behavior rating (DBR) has been described as a hybrid of systematic direct observation and behavior rating scales. Although single-item (DBR-SIS) and multi-item (DBR-MIS) methods have been advocated, the overwhelming majority of research attention has focused on DBR-SIS. This study employed generalizability theory to compare the dependability of the two DBR methods for two behaviors (Acad...

This study examines the definitions of theory and the implications axiomatic presuppositions have on theorybuilding research. Theory-building is important because it provides a framework for analysis, facilitates the efficient development of the field, and is needed for the applicability to practical real-world problems. To be good theory, a theory must follow the virtues (criteria) for ‘good’ ...