The purpose of the current study was to analyze the relationship between automated essay scoring (AES) and human scoring in order to determine the validity and usefulness of AES for large-scale placement tests. Specifically, a correlational research design was used to examine the correlations between AES performance and human raters’ performance. Spearman rank correlation coefficient tests were...

The e-rater system 1 , an automated essay scoring system, developed at Educational Testing Service (ETS), is currently being used to score essay responses on the Graduate Management Admissions Test (GMAT). The average agreement between human readers, and between independent human readers and e-rater is approximately 92%. There is much interest in the larger writing community in examining the sy...

James BEEBE presents a model for doing what he labels the Rapid Assessment Process (RAP). This review essay provides a critical review of and commentary on BEEBE's text for qualitative researchers to use for applied evaluation projects. The RAP model is a systematic approach based upon collaborative inclusion of insider and outsider perspectives in building a research team that collectively con...

Neural approaches to automated essay scoring have recently shown state-of-theart performance. The automated essay scoring task typically involves a broad notion of writing quality that encompasses content, grammar, organization, and conventions. This differs from the short answer content scoring task, which focuses on content accuracy. The inputs to neural essay scoring models – ngrams and embe...

We present the first system developed for automated grading of high school essays written in Swedish. The system uses standard text quality indicators and is able to compare vocabulary and grammar to large reference corpora of blog posts and newspaper articles. The system is evaluated on a corpus of 1 702 essays, each graded independently by the student’s own teacher and also in a blind re-grad...

Research on automated essay scoring (AES) indicates that computer-generated essay ratings are comparable to human ratings. However, despite investigations into the accuracy and reliability of AES scores, less attention has been paid to the feedback delivered to the students. This paper presents a method developers can use to quickly evaluate the usability of an automated feedback system prior t...

Chinese automated essay scoring (CAES) is a very important tool for many educational researches. However, none of the methods for retrieving features in English essays is applicable to Chinese writings because Chinese grammar parsers cannot produce reliable and useful syntactic features. CAES systems must explore other distinct features in Chinese writing. This paper proposes a method for retri...

In this autobiographical essay I reflect on my training in linguistics and the way it affected my interpretation and development of SFL theory. In particular I am concerned to show how I tried to help SFL evolve, accumulating previous understandings into a model with additional theoretical architecture taking descriptive responsibility for a wider range of linguistic data. This evolution is ill...

This essay addresses the power of formal analysis, and exemplifies some of the important issues that arise when we use formal analyses to address the substantive issues of kinship terminologies – and, through them, of category systems in general. These examples point to where a broad empirical understanding of kin terminologies can take us and indicate something of the potential import of kinsh...

Quantitative social science is not only about regression analysis or, in general, data inference. Computer simulations of social mechanisms have an over 60 years long history. They have been used for many different purposes—to test scenarios, to test the consistency of descriptive theories (proofof-concept models), to explore emerging phenomena, for forecasting, etc. In this essay, we sketch th...