The statistical analysis technique of correlation has extensively been used in second/ foreign language testing research for a variety of purposes. Most commonly, the technique has been used to validate newly constructed language tests (particularly cloze tests) against previous supposedly valid measures. Tests such validated are meant to substitute the older ones, and based on these newly vali...

The LSDSem’17 shared task is the Story Cloze Test, a new evaluation for story understanding and script learning. This test provides a system with a four-sentence story and two possible endings, and the system must choose the correct ending to the story. Successful narrative understanding (getting closer to human performance of 100%) requires systems to link various levels of semantics to common...

We present an internet-based system that helps teachers to make a cloze test on an online news article. The current system supports question making on grammar and vocabulary of English. The system works as an assistant to the teacher. It helps the user to choose an article, highlights grammar targets, suggests possible choices for the wrong alternatives, and formats the questions in a printer-f...

This paper presents a pilot system and discusses its possible extensions. In the first sections, we present a web-based test authoring system for English grammar and vocabulary. It assists language teachers to make questions on arbitrary input texts, such as online news. The system employs the techniques of NLP (Natural Language Processing) and is compared with other studies in the same field. ...

The Story Cloze Test consists of choosing a sentence that best completes a story given two choices. In this paper we present a system that performs this task using a supervised binary classifier on top of a recurrent neural network to predict the probability that a given story ending is correct. The classifier is trained to distinguish correct story endings given in the training data from incor...

This paper analyzes the narrative event cloze test and its recent evolution. The test removes one event from a document’s chain of events, and systems predict the missing event. Originally proposed to evaluate learned knowledge of event scenarios (e.g., scripts and frames), most recent work now builds ngramlike language models (LM) to beat the test. This paper argues that the test has slowly/un...

this study aimed at investigating differential item functioning (dif) on the special english test of the iranian national university entrance exam (inuee). the effect of gender and subject area was taken into account. the study utilized one-parameter irt model with a sample of 36000 students who sat for the inuee special english test in 2004 and/or 2005. the findings confirmed the presence of d...