Nowadays crowdsourcing is widely used in supervised machine learning to facilitate the collection of ratings for unlabelled training sets. In order to get good quality results it is worth rejecting results from noisy/unreliable raters, as soon as they are discovered. Many techniques for filtering unreliable raters rely on the presentation of training instances to the raters identified as most a...

OBJECTIVE Interrater agreement and construct validity of the Revised Knox Preschool Play Scale (RKPPS) were examined. METHOD Two separately trained raters evaluated 38 typically developing children, ages 36 to 72 months. For each child, the raters observed two 15-min free-play sessions. RESULTS For the overall play age, the scores of the two raters were within 8 months of each other 86.8% o...

PURPOSE In patients with multiple sclerosis (MS), Double Inversion Recovery (DIR) magnetic resonance imaging (MRI) can be used to identify cortical lesions (CL). We sought to evaluate the reliability of CL detection on DIR longitudinally at multiple subsequent time-points applying the MAGNIMs scoring criteria for CLs. METHODS 26 MS patients received a 3T-MRI (Siemens, Skyra) with DIR at 12 ti...

To what extent do people achieve accuracy in judging others’ situations? Based on interpersonal perception models, we propose that ex-situ raters may attain accuracy by judging the psychological characteristics of a situation that in-situ raters have experienced according to a normative and distinctive characteristics profile. Biesanz’ Social Accuracy Model (SAM) provides a flexible crossed-eff...

In order to assess the reliability of a given characterization of a subject it is often necessary to obtain multiple readings, usually but not always from different individuals or raters. The degree of agreement among the various raters gives some indication as to the consistency of the values. If agreement is high, we feel more confident the ratings reflect the actual circumstance. If agreemen...

Agile Software Development methodologies have grown in popularity both among academic researchers and industrial practitioners. Among the various methodologies or practices proposed, pair programming, which is concerned with two programmers collaborating on design, coding and testing, has become a controversial focus of interest. Even though some success stories have been reported with the use ...

conditions and quality of ethical decision making by entrepreneurs is an important concern in academic and public area in iran. different factors such as personal, environmental and nature of business are causing the ethical decision making by entrepreneurs, but the concern have to great attention in iranian academic institutions. thus, research goal is try to identify environmental factors inf...