Quality of Residents’ Promotion Exams before & after Educational Intervention in 2010-2011

Introduction: Resident promotion exams are considered one of the most important summative exams in residency period that allow residents to proceed to a higher grade. Multiple choice questions are the most common and objective test in medical science education. Therefore, finding methods to improve quality of exams, particularly in specialty programs, is important. The aim of this study is to determine effectiveness of training workshops on improving quality of residency promotion exams in medical school of Guilan University of Medical Sciences (GUMS). Methods: In this Quasi-experimental study (before & after), all questions of resident promotion exams of GUMS for 12 specialty programs in year 2010 (150 questions for each specialty-1800 questions in total) were structurally and taxonomically evaluated using a valid and reliable checklist based on Millman principles. After12 one–day educational workshops (one for each department) on multiple choice questions), questions of residents’ promotion exam in year 2011 were evaluated. Data related to questions designed by faculties who had participated in the interventional workshop were analyzed using statistical Wilcoxon signed-rank test. Results: Taxonomy of questions of written promotion exam in years 2010 and 2011 were as follows: 1204(%66.09) questions were of first level taxonomy, 565 (%31.4) of 2nd level, and 31(% 1.7) of 3rd level in the 2010 exam, and 867 questions (%48.2) were of first level taxonomy, 741 (%41.2) of 2nd level, and 192 (%10.6) of 3rd level in the 2011. Findings showed significant differences between taxonomy of questions of different programs in 2010 and 2011 (p=0.0001, Z=23.83). Also, according to Millman’s structural principles, the rate of questions with no structural problem was %66.77 in the year 2010 and %75.22 in 2011. There is a statistically significant difference based on Millman principles between 2010 and 2011 (p=0.0001, Z=16.703). Conclusion: Current study showed that training workshops are effective in improving the rate of questions targeted at higher levels of knowledge (Taxonomies 2 & 3) in residency promotion exams.