We describe the automated generation and use of 69,326 comprehension cloze questions and 5,668 vocabulary matching questions in the 2001-2002 version of Project LISTEN’s Reading Tutor used by 364 students in grades 1-9 at seven schools. To validate our methods, we used students’ performance on these multiple-choice questions to predict their scores on the Woodcock Reading Mastery Test. A model ...

Can automatically generated questions scaffold reading comprehension? We automated three kinds of multiple-choice questions in children’s assisted reading: 1. Whquestions: ask a generically worded What/Where/When question. 2. Sentence prediction: ask which of three sentences belongs next. 3. Cloze: ask which of four words best fills in a blank in the next sentence. A within-subject experiment i...

In the Story Cloze Test, a system is presented with a 4-sentence prompt to a story, and must determine which one of two potential endings is the ‘right’ ending to the story. Previous work has shown that ignoring the training set and training a model on the validation set can achieve high accuracy on this task due to stylistic differences between the story endings in the training set and validat...

This paper examines the effect of gender and type of stem on Iranian test takers’ performance on the grammar section of a nation-wide language proficiency test. In so doing, the scores of 2931 examinees (1107 female and 1824 male) who sat Tarbiat Modares English proficiency test were obtained. The examinees’ scores on three types of MC grammar items with different kinds of stems (i.e., blank fi...

The narrative cloze is an evaluation metric commonly used for work on automatic script induction. While prior work in this area has focused on count-based methods from distributional semantics, such as pointwise mutual information, we argue that the narrative cloze can be productively reframed as a language modeling task. By training a discriminative language model for this task, we attain impr...

It has been suggested that the right hemisphere (RH) has a privileged role in the processing of figurative language, including metaphors, idioms, and verbal humor. Previous experiments using hemifield visual presentation combined with human electrophysiology support the idea that the RH plays a special role in joke comprehension. The current study examines metaphoric language. Event-related pot...

Language proficiency tests are a useful tool for evaluating learner progress, if the test difficulty fits the level of the learner. In this work, we describe a generalized framework for test difficulty prediction that is applicable to several languages and test types. In addition, we develop two ranking strategies for candidate evaluation inspired by automatic solving methods based on language ...