Can we decrease the costs of database curation by crowd-sourcing curation work or by offloading curation to publication authors? This perspective considers the significant experience accumulated by the bioinformatics community with these two alternatives to professional curation in the last 20 years; that experience should be carefully considered when formulating new strategies for biological d...

A large part of empirical social science relies heavily on data that are not observed in the field, but are generated by researchers sitting at their desks, raising obvious issues of both reliability and validity. This paper addresses these issues for a widely used type of coded data, derived from the content analysis of political text. Comparing estimates derived from multiple “expert” and cro...

Crowd-sourcing is increasingly being used for providing answers to online polls and surveys. However, existing systems, while taking care of the mechanics of attracting crowd workers, poll building, and payment, provide little that would help the survey-maker or pollster to obtain statistically significant results devoid of even the obvious selection biases. This paper proposes InterPoll, a pla...

Obtaining good pronunciations for named-entities poses a challenge for automated speech recognition because namedentities are diverse in nature and origin, and new entities come up every day. In this paper, we investigate the feasibility of learning named-entity pronunciations using crowd-sourcing. By collecting audio samples from non-linguistic-expert speakers with Mechanical Turk and learning...

Public sensing is a new concept for gathering sensor data by utilizing ubiquitously available mobile devices such as smartphones. By leveraging the crowd of mobile devices, we can build a mobile sensor network with excellent coverage in urban areas without large up-front investments for building a dedicated sensor network. In this paper, we present several concepts and methods to improve the us...

In order to gain a better understanding of users and their intent behind web searching activities, first steps involve the analysis of the query submitted by the user and correct categorical classification of the query as a input for further analyses. Natural Language Processing (NLP) is an area with many inaccuracies for problem areas in Word Sense Disambiguation (WSD) and terms detected that ...

Labeled data is a prerequisite for successfully applying machine learning techniques to a wide range of problems. Recently, crowdsourcing has shown to provide effective solutions to many labeling tasks. However, tasks in specialist domains are difficult to map to Human Intelligence Tasks (or HITs) that can be solved adequately by "the crowd". The question addressed in this paper is whether thes...

© Understanding the readability of colored text by crowd-sourcing on the Web Silvia Zuffi, Carla Brambilla, Giordano Beretta, Paolo Scala HP Laboratories HPL-2009-182 color; psychophysics We present an experiment about the readability of colored text on a colored background that we have conducted through crowd-sourcing on the Web. Our aim is to contribute to the understanding of the ease of rea...

Opinions about the 2016 U.S. Presidential Candidates have been expressed in millions of tweets that are challenging to analyze automatically. Crowdsourcing the analysis of political tweets effectively is also difficult, due to large inter-rater disagreements when sarcasm is involved. Each tweet is typically analyzed by a fixed number of workers and majority voting. We here propose a crowdsourci...