In this paper, we describe a holistic annotation scheme for coherence quality that requires little expertise on the part of the human annotator; we also present computational systems for evaluating coherence quality in essays. A formidable challenge to reliability when annotating discourse coherence comes from differences among annotators in the inferences that they draw when reading an essay. ...

In this study we evaluate the ability of a state-of-the-art speechto-speech machine translation system, Skype Translator, to preserve the communicative effectiveness of its input speech. Specifically, we elicited Spanish spoken responses from 24 native speakers of Spanish in the following two scenarios: academic speaking (in the context of 6 questions from a standardized assessment of academic ...

Soil contamination is one of the major threats constraining proper functioning of the soil and thus provision of ecosystem services. Remedial actions typically only address the chemical soil quality by reducing total contaminant concentrations to acceptable levels guided by land use. However, emerging regulatory requirements on soil protection demand a holistic view on soil assessment in remedi...

Background Outcome measures typically assess single domains making holistic assessment difficult. Our purpose was to develop a mobility composite measure (MCM) based on four commonly used outcome measures and compare this composite score to the individual measures in patients with neurologic disorders. Methods We retrospectively reviewed 148 medical records for inclusion of primary neurologic...

Multidimensional and dynamic talent models represent the current state of the art, but these demands have hardly ever been implemented so far. One reason for this could be the methodological problems associated with these requirements. This paper will present a proposal for dealing with this, namely for examining the development of young soccer players holistically. The patterns formed by the c...

This study compared the results from nine automated essay scoring engines on eight essay scoring prompts drawn from six states that annually administer high-stakes writing assessments. Student essays from each state were randomly divided into three sets: a training set (used for modeling the essay prompt responses and consisting of text and ratings from two human raters along with a final or re...

ABSTRACT: Using the generalizability theory (G-theory) as a theoretical framework, this study aimed at investigating variability and reliability of holistic scores assigned by human raters e-rater to same EFL essays. Eighty argumentative essays written on two different topics tertiary level Turkish students were scored holistically eight who received detailed rater training. The results showed ...

This paper describes an innovative model for assessing the technical communication course by analytically scoring online portfolios, open to the internet, for ten separate (analytic) variables and one overall (holistic) score. The model is a statistically verifiable and sustainable method that strengthens the curriculum and fosters consensus within the teaching community. We achieved construct ...