Surveys are widely used across the social sciences, business, and even the natural sciences to draw conclusions about human behavior. When important, high-stakes decisions are being made on the basis of such tools as surveys, we hope that the inferences that lead to those decisions are sound.

Unfortunately, bugs creep into surveys, much in the same way that bugs creep into computer programs: authors of surveys neglect edge cases, they introduce bias, and they fail to consider how the survey will be received in the wild. Many of these bugs are well-known to those who use surveys regularly in their research. Practitioners handle these bugs by running pilot studies and performing post-hoc statistical analyses on the results obtained, ensuring the quality of their responses.

Unfortunately, there are some bugs that are not as widely known, and there are some bugs that cannot be diagnosed through post-hoc analyses. SurveyMan addresses these problems by controlling the experimental setup of a survey. By automating large portions of the deployment and debugging of surveys, SurveyMan allows the experimenter to conduct reliable, repeatable surveys on the web at scale.

• We released a technical report in March. This draft has been accepted to OOPSLA 2014.
• Check out our slides from OBT 2014!
• For regular updates on our upcoming Python library, see @mmcmahon13‘s blog. For general musings on SurveyMan research, see @etosch‘s blog.

Documentation

Follow the SurveyMan organization on github! You can find documentation for our individual libraries here:

• Language Parser, Static Analyser, Dynamic Analyser, Deployment Backend, Debugger
• Browser-based Runtime System
• JSON Schema for communicating across components
• Python interface to the SurveyMan language

Authors and Contributors

SurveyMan is a collaborative research project between the departments of Computer Science and Linguistics at the University of Massachusetts Amherst. The code is primarily the work of @etosch. Other student collaborators on code and research are @mmcmahon13 and @presleyp. @emeryberger is the principal advisor, with @jpater providing input from Linguistics.