Constructing and Validating a Q-Matrix for Cognitive Diagnostic Analysis of a Reading Comprehension Test Battery

Authors

Abstract:

Of paramount importance in the study of cognitive diagnostic assessment (CDA) is the absence of tests developed for small-scale diagnostic purposes. Currently, much of the research carried out has been mainly on large-scale tests, e.g., TOEFL, MELAB, IELTS, etc. Even so, formative language assessment with a focus on informing instruction and engaging in identification of student’s strengths and weaknesses to guide instruction has not been conducted in the Iranian English language learning context. In an attempt to respond to the call for developing diagnostic tests, this study explored developing a cognitive diagnostic reading comprehension test for CDA purposes. To achieve this, initially, a list of reading attributes was prepared based on the literature and then the attributes were used to construct 20 reading comprehension items. Then seven content raters were asked to identify the attributes of each item of the test. To obtain quantitative data for Q-matrix construction, the test battery was administered to 1986 students of a General English Language Course at the University of Tehran, Iran. In addition, 13 students were recruited to participate in think-aloud verbal protocols. On the basis of the overall agreement of the content raters’ judgements concerning the choices of attributes and results of think-aloud verbal protocol analysis, a Q-matrix that specified the relationships between test items and target attributes was developed. Finally, to examine the CDA of the test, the Fusion Model, a type of cognitive diagnostic model (CDM), was used for diagnosing the participants' strengths and weaknesses. Results suggest that nine major reading attributes are involved in these reading comprehension test items. The results obtained from such cognitive diagnostic analyses could be beneficial for both teachers and curriculum developers to prepare instructional materials that target specific weaknesses and inform them of the more problematic areas to focus on in class in order to plan for better instruction.

Upgrade to premium to download articles

Sign up to access the full text

Already have an account?login

similar resources

designing and validating a textbook evaluation questionnaire for reading comprehension ii and exploring its relationship with achievement

در هر برنامه آموزشی، مهم ترین فاکتور موثر بر موفقیت دانش آموزان کتاب درسی است (مک دونو و شاو 2003). در حقیقت ، کتاب قلب آموزش زبان انگلیسی است( شلدن 1988). به دلیل اهمیت والای کتاب به عنوان عنصر ضروری کلاس های آموزش زبان ، کتب باید به دقت ارزیابی و انتخاب شده تا از هرگونه تاثیر منفی بر دانش آموزان جلوگیری شود( لیتز). این تحقیق با طراحی پرسش نامه ارزیابی کتاب که فرصت ارزیابی معتبر را به اساتید د...

15 صفحه اول

using c-test passages for improving reading comprehension ability and incidentally learned vocabulary of iranian efl learners

this study was an attempt to investigate the effect of using c-test passages on the reading comprehension and incidental vocabulary learning of iranian intermediate efl learners. the participants were 60 male efl learners at kish mehr institute in garmsar. in fact, there were two groups, an experimental group and a comparison group with 30 students in each. the participants were pretested throu...

15 صفحه اول

passivity in waiting for godot and endgame: a psychoanalytic reading

this study intends to investigate samuel beckett’s waiting for godot and endgame under the lacanian psychoanalysis. it begins by explaining the most important concepts of lacanian psychoanalysis. the beckettian characters are studied regarding their state of unconscious, and not the state of consciousness as is common in most beckett studies. according to lacan, language plays the sole role in ...

the relationship between field-related cognitive styles and global reading comprehension of hypertext

this study investigates the relationship between field-related cognitive styles and global reading comprehension of hypertext. technically, field dependency is prevailing dominance of field on perception of parts, while global comprehension is skim-like reading to grasp the general content. in this research project, 60 senior and junior students of arak university majoring in english literature...

15 صفحه اول

‏‎a phonological contrastive analysis of kurdish and english‎‏

deposite the different criticisms on contrastive analysis it has been proved that the results of it(when processed)can be usuful in a tefl environment,specially at the level of phonology.this study is an attempt to compare and contrast the sound systems of kurdish and english for pedagogical aims. the consonants,vowels,stress and intonation of the twolanguages are described by the same model-ta...

15 صفحه اول

the effects of cognitive strategies- note making and underlining- on iranian efl learners’ reading comprehension

هدف از انجام این مطالعه ، بررسی اثرات استفاده از استراتژی های شناختی یعنی "نت برداری" و "زیر خط کردن" بر روی عملکرد درک مطلب زبان آموزان ایرانی می باشد. به این منظور، 60 زبان آموز دختر سال چهارم دبیرستان از طریق آزمون پیشرفت زبان انگلیسی (نلسون)، از یک جامعه آماری بزرگتر انتخاب شدند. سپس به صورت تصادفی به سه گروه که هر گروه شامل ?0 دانش آموز همسان بود، تقسیم شدند: دو گروه آزمایش(گروه "نت برداری...

15 صفحه اول

My Resources

Save resource for easier access later

Save to my library Already added to my library

{@ msg_add @}


Journal title

volume 10  issue 21

pages  1- 31

publication date 2018-05-22

By following a journal you will be notified via email when a new issue of this journal is published.

Hosted on Doprax cloud platform doprax.com

copyright © 2015-2023