Committee-designed languages such as those of the OMG consortium are widely used in both industry and academia. These languages seem to be used increasingly by users with no technical background for the visualization, documentation and specification of workflows, data and software systems. However, according to several studies on these languages, the used visual notations do not seem to convey ...

Commonsense is a challenge not only for representation and reasoning but also for large scale knowledge engineering required to capture the breadth of our ‘everyday’ world. One approach to knowledge engineering is to ‘outsource’ the effort to the public through games that generate structured commonsense knowledge from user play. To date, such games have focused on symbolic and textual knowledge...

Searching for concepts in science and technology is often a difficult task. To facilitate concept search, different types of human-generated metadata have been created to define the content of scientific and technical disclosures. Classification schemes such as the International Patent Classification (IPC)1 and MEDLINE's MeSH2 are structured and controlled, but require trained experts and centr...

In large-scale proteomics studies there is a temptation, after months of experimental work, to plug resulting data into a convenient-if poorly implemented-set of tools, which may neither do the data justice nor help answer the scientific question. In this paper we have captured key concerns, including arguments for community-wide open source software development and "big data" compatible soluti...

Participatory sensing, when a crowd of users collaborate to collect useful information, based applications are getting popular these days thanks to the proliferation of powerful mobile devices. The builtin sensors of smartphones offer an easy and handy way to monitor the environment and collect data which can serve as the basis of smart applications. However, the quick and flexible development ...

Human evaluation of machine translation quality is a key element in the development of machine translation systems, as automatic metrics are validated through correlation with human judgment. However, achievement of consistent human judgments of machine translation is not easy, with decreasing levels of consistency reported in annual evaluation campaigns. In this paper we describe experiences g...

C ollecting and visualizing accessibility information are always challenging problems within the realm of Human-Computer Interaction (HCI). A team of researchers at the University of Maryland have proposed innovative approaches that combine crowdsourcing, computer vision and machine learning to detect and visualize how accessible a particular street or sidewalk is. Tohme is one such approach fo...

Over a series of evaluation experiments conducted using naive judges recruited and managed via Amazon’s Mechanical Turk facility using a task from information retrieval (IR), we show that a SVM shows itself to have a very high accuracy when the machine-learner is trained and tested on a single task and that the method was portable from more complex tasks to simpler tasks, but not vice versa.

Applications extracting data from crowdsourcing platforms must deal with the uncertainty of crowd answers in two different ways: first, by deriving estimates of the correct value from the answers; second, by choosing crowd questions whose answers are expected to minimize this uncertainty relative to the overall data collection goal. Such problems are already challenging when we assume that ques...