Accounting for Peer Reviewer Bias with Bayesian Models

Instructors and researchers of peer review would benefit from a consistent way of characterizing peer review among students. One factor that can affect peer review is reviewer bias. For example, students may give biased assessments if some reviewers are lenient and others stringent. Accordingly, statistical models of peer review should account for reviewer bias. We present work in progress comparing alternative models of bias, including models that postulate that bias may be a multidimensional construct and models that consider whether bias is a useful predictor of instructor assessment. We find that a model that represents bias as multidimensional and as predicting instructor assessment is the best-fitting model on one of two real-world datasets.