Online Experiments using jsPsych, psiTurk, and Amazon Mechanical Turk

نویسندگان

  • Josh de Leeuw
  • Anna Coenen
  • Douglas Markant
  • Jay B. Martin
  • John V. McDonnell
  • Alexander Rich
  • Todd M. Gureckis
چکیده

This half-day tutorial will cover how to build and deploy online experiments using jsPsych, psiTurk, and Amazon Mechanical Turk (AMT). jsPsych is an open-source JavaScript library that facilitates building behavioral experiments in a web browser. psiTurk is an open-source Python platform that simplifies the process of running an experiment using AMT. Together, these two software packages reduce the complexity of setting up an online experiment on AMT, enabling researchers with little software programming experience to take advantage of online experiments. By the end of the tutorial, participants will have gained hands-on experience in programming and deploying a basic behavioral experiment on AMT. Researchers in the Cognitive Science community have been using AMT, and online experiments in general, for several years, but the learning curve can be steep for researchers who are not familiar with web development. While some tools exist for certain kinds of simple experiments (e.g. questionnaires), programming more complex experiments with dynamic elements requires knowledge of web-oriented programming. The tools covered in this tutorial simplify the process of programming online experiments, opening up the possibility of conducting online experiments to more researchers in the cognitive science community. Workshops covering AMT (Mason & Suri,2011) and psiTurk (Coenen, Markant, Martin, & McDonnell,2013) have been offered at previous Cognitive Science Society meetings. This tutorial goes one step further by covering jsPsych as well. Together, these tools cover the entire process of assembling and running an online experiment.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Using Mechanical Turk and PsiTurk for Dynamic Web Experiments

This half-day workshop will demonstrate how to build custom web-based experiments that rely on participants from Amazon Mechanical Turk (AMT). Attendees will learn how to deploy web-based experiments using PsiTurk, a Pythonbased platform that simplifies the process of setting up experiments and interacting with AMT. Workshops discussing the AMT marketplace have been offered at previous Cognitiv...

متن کامل

Running experiments on Amazon Mechanical Turk

Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality of data provided by subjects recruited from online labor markets. We address these potential concerns by presenting new demographic data about the Mechanical Turk subject population, reviewing the strengths of Mechanical Turk relative to other online...

متن کامل

TurkServer: Enabling Synchronous and Longitudinal Online Experiments

With the proliferation of online labor markets and other social computing platforms, online experiments have become a low-cost and scalable way to empirically test hypotheses and mechanisms in both human computation and social science. Yet, despite the potential in designing more powerful and expressive online experiments using multiple subjects, researchers still face many technical and logist...

متن کامل

psiTurk: An open-source framework for conducting replicable behavioral experiments online.

Online data collection has begun to revolutionize the behavioral sciences. However, conducting carefully controlled behavioral experiments online introduces a number of new of technical and scientific challenges. The project described in this paper, psiTurk, is an open-source platform which helps researchers develop experiment designs which can be conducted over the Internet. The tool primarily...

متن کامل

The (Statistical) Power of Mechanical Turk

In this paper, I argue for the use of Amazon Mechanical Turk (AMT) in language research. AMT is an online marketplace of paid workers who may be used as subjects, which can greatly increase the statistical power of studies quickly and with minimal funding. I will show that—despite some obvious limitations of using distant subjects—properly designed experiments completed on AMT are trustworthy, ...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2014