1. gureckislab

  2. collecting behavioral data online

  3. Syllabus Spring 2014

Spring 2014

Syllabus

This is a graduate course in the psychology department at NYU. The following document describes the policies and grading proceedure. Everything is subject to change.

Meeting time/place

Wednesdays from 12-1:50pm in Meyer 851.

Overview

The aim of this seminar course is explore using the Internet to collect behavioral data. The course is structured to be very hands-on. Students will be expected to program an experiment relevant to their research goals by the end of the class.

The course will be taught using a "flipped classroom" design. The lectures and readings will be posted on this webpage before class. Students will be expected to watch the videos and follow along with the instruction before they come to class. In class time will be spent entirely in constructive conversation, debugging, talking with other students about problems that came up, etc... The idea is that you can get the lecture stuff offline on your own time/pace but in class time it is more valueable to get hands-on help from other students and from the professor (me!).

Blurb

Online data collection is revolutionizing many aspects of experimental psychology by allowing access to large and (potentially) diverse participant populations. Online collection may be useful for many creative research applications including stimulus generation/norming, crowd-based computation (e.g., obtaining independent coding or raters for laboratory collected data), group experiments that require multiple participants to interact simultaneously, or simply running standard experiments quickly and with large sample sizes. However, the skills required to successfully manage an online experiment are somewhat more complex than a traditional experiment design. For example, researchers need to have an understanding of web servers, databases, encryption, cross-browser web programming, etc.. The goal of this course is to teach students in a hands-on way how to collect data online. Topics covered include how to maximize the quality of online data collection, how to design dynamic experiments that run in a browser, the limits to what browsers can record in terms of reaction time or stimulus timing, and how to maximize the chance that participants can view your experiment successfully. The goal will be to have everyone in class make an online experiment useful for their research by the end of the semester. Specific skills taught include the basics of Javascript, databases/SQL, Amazon's Mechanical Turk, and a bit of Python. Gaining basic proficiency with these topics will not only increase your overall research productivity but also are also in heavy demand in industry.

Computer requirement

Everyday in class we will be working on computers and the Internet. You will need a laptop to take this course because it is not being held in a computer lab (the NYU computer lab does not have Internet access). In an extreme case a loaner laptop may be made available however this is not an ideal situation since you won't be able to keep your system setup to work at home. In addition, it is preferred that you use Mac OS X or Linux/Unix. Windows will have considerable problems with the software. If you really must use windows I will help you set up with a remote unix server that you can log into. Note, acceptable Linux-based laptops can be had for a little as $150-200 (similar to the cost of some textbooks). I'm sorry if this requirement is a pain or seems "politically" insensitive, but if you are serious about getting online data a unix-based system is a much better place to start.

Grading

Active class participation (15%),various assignments/exercises (15%), final project (70%). This is a learn-by-doing class. What you put in you will get out (specifically, an experiment you can run online with hundreds or thousands of participants).

Textbook

There is no textbook for the course. Many handouts, reading, blog posts, and other resources will be shared each week. Students are expected to read this material before class.

Time commitment

This class may involve a significant time commitment. All online experiment work does because it involves programming which can (at times) be fustrating and result in hours of lost time due to small bugs/typos. Do not take this course if you do not really want to put together an online experiment for your research. It just won't be worth it if you only have a passing interest in online experimentation.

Schedule

The class schedule will roughly follow the outline here. Note there are fewer planned lectures than there are class meeting times. This is to accomodate the inevitable fact that things may blur together or run over from time to time.

Instructor

Todd Gureckis. I'm a computational cognitive scientists who does a lot of online data collection these days. My office is Meyer 859. Office hours by appointment.