For more information

For more information regarding setting up a Qualtrics survey, see the Library's webpage: Qualtrics

Online Surveys

While online surveys often qualify as being exempt under Exempt Category 2, remember that there are qualifications involved in both this category and HRPP policy. Always contact the IRB administrator if you are unsure whether your survey is, indeed, human subjects research, and whether you may submit an Exempt Research Protocol in lieu of an Initial Research Protocol. Note that, no matter which application is submitted, a copy of the complete survey must be included.

Expand All / Collapse All

The survey platform

The Brandeis University preference and default is that investigators use Qualtrics when conducting online surveys.

Qualtrics is a cloud-based survey tool for which Brandeis University has a license for all faculty, staff, and students. Qualtrics has been vetted by Brandeis University’s IT department and is considered a secure system appropriate for all data, including sensitive data and PHI. 

Limiting the number of questions

Limit your questions to those that are truly relevant to your research question. Eliminating questions that may be interesting but aren't really necessary to answer your research question is important not only to keep your survey as short as possible (which may reduce dropouts and simplify your data analysis), but for issues of privacy and confidentiality.

Keep in mind the more questions you ask, the easier it will be to identify subjects and link them to their answers. This includes questions that may seem innocuous - even demographic questions. For example: You survey the student body and ask your subjects to provide you with their major, sex, and ethnicity. Your response rate is good - 70%. Still, only one student identified themselves as a Language and Linguistics major, male, and Hispanic/Latino.

The difference between anonymous and confidential

Not all surveys are anonymous! The terms anonymous and confidential are often confused and many investigators promise their subjects that a survey will be anonymous when, in actuality, it is not.

Anonymity can be promised only when it is impossible for anyone (including the PI) to link the survey answers to any information that could potentially be used to identify or trace a specific subject. And as discussed above, demographic variables, or even the answer to one simple question on a survey, can potentially be used to link the entire completed survey to a single individual.

Some things to keep in mind: use as large a sample, as diverse a sample, and as few demographic questions as you can. And, think carefully about all potential identifiers - one identifier that is often overlooked when conducting surveys online, for example, is computer IP addresses.

Note that Qualtrics' default is to collect IP address; however, you can remove them by enabling Anonymize Responses in the Survey Options.

Confidentiality, on the other hand, refers to the expectation that a subject's information will not be divulged to others without his/her permission, and that you, as the investigator, promise to do all you can to keep the subject's information from being discovered. Because anonymity can be difficult to achieve, most research conducted is considered confidential.

Electronic informed consent

Informed consent is the process by which you, the investigator, explain your research to your prospective subjects so that they understand the research, the possible consequences of participating, and that their (ongoing) participation is completely voluntary.

When conducting a survey in person, the subject is usually asked to sign a consent form. When conducting a survey online, informed consent is usually obtained electronically.

Note that E-signatures are permissible provided the subject’s identity is verifiable. If the subject’s identity is not verifiable, a request for a waiver of documented informed consent must be requested.

Online surveys usually begin with a screen display with the information that would be contained in a traditional consent form, and a checkbox that indicates the subject's acknowledgment that s/he has read the information provided and consents to participate.

For information on how to construct an informed consent form, see the Informed Consent page of the HRPP website.

Consider the risks and possible harm to subjects

When conducting research, there is always a risk of negative effects, such as psychological distress from being asked questions they aren't comfortable with, and the potential for a breech of confidentiality and its possible ramifications (such as legal or social). Prospective subjects should always be warned about these possible harms during the informed consent process and given the opportunity to decline to participate in the research or skip questions they don't want to answer (be sure this is an option with your survey software).

Consider the risks to subjects as you develop your survey questions. Are you asking personal or sensitive questions - and if so, are they truly necessary to answer your research question? Be mindful of the fact that when conducting research online, you cannot gauge the reaction of your subjects (such as if they find the questions stressful).

If your questions have the potential for harm - especially if more than minimal (where the probability and magnitude of harm or discomfort is greater that that ordinarily encountered in daily life or psychological examinations or tests), include a debriefing at the end of the survey with information on whom they can contact for help, if they wish (such as counseling services or online support groups).

Compensating your subjects

You may choose to compensate subjects for the time and effort they put into completing your survey. When compensating your subjects there are a few things to be aware of:

First is how much - and whether you pay each subject the same amount or use a raffle, where each subject has an equal chance to win something. Inherent in this is what the compensation will be - cash, a gift certificate, or an object such as an iPad. Note that the use of a gift certificate may allow you to compensate your subjects without their having to supply their contact information, allowing you to keep your survey anonymous or help support confidentiality.

Second is whether they will be compensated only if they complete the survey, or receive the compensation even if they withdraw before they complete it. If compensation is not contingent on completion of the survey you must include a way for the subjects to collect their compensation. For example, you may want to include a button that will take them to the end of the survey where they can pick up their gift certificate or provide their contact information.

Third, if collecting their contact information, always have the survey bring the subject to a new page, separate from the survey, unless the survey itself requires your subjects to identify themselves in order for the data to be accepted.

Protecting your data

One risk you can never avoid with surveys that are not anonymous is a breach of confidentiality. Two things you should consider are 1) limit who has access to the data, and 2) use the highest level of security possible - be sure your server is secure and your data encrypted, for example. Brandeis' policy is to use whenever possible. If you have any questions regarding data security, contact the Brandeis University security office.

Special considerations when using MTurk

MTurk is becoming a popular platform for conducting research - it's thought to be cheap, easy, anonymous, and to reach a large and diverse population. Unfortunately, however, MTurk does not come without its problems.

Before using MTurk to conduct your survey, be sure to read Amazon's MTurk privacy policy and note that has stated that the MTurk platform is NOT meant to support subject anonymity: MTurk worker IDs are linked to workers' profiles, often allowing one to see such things as the subject's buying habits, wish lists, video tastes, product reviews, and sometimes names and photos. In addition, MTurk automatically collects subjects’ IP addresses, which can be potentially identifiable, depending on context. 

For this reason, MTurk should only be used as a recruitment tool with a survey link embedded within the MTurk task that redirects your subjects to complete the survey at an external site such as Qualtrics. Have Qualtrics assign your subjects unique completion codes which can then be provided in the MTurk task in order for the subjects to be paid through MTurk. 

If you must collect your subjects’ worker IDs, these should be de-identified as soon as is feasible. This process must be explained in your IRB application and mentioned in your informed consent page.

Some additional things to consider when using MTurk as a recruitment tool:

  • Consider the appropriateness of your HIT title - keep in mind this is how you are recruiting subjects
  • Include any screening questions on the Preview page or use a Qualification Test to screen prospective subjects - or compensate for completing a screening survey whether or not the subject qualifies for your research project

  • Include a description of the research on the Preview page - what will subjects be asked to do and how long will it take (be sure to be reasonable - over-estimate rather than under-estimate)?

  • Include basic consent form information on the Preview page
  • Compensate fairly - pay should be equivalent to what your subjects would be paid if you were using any other recruitment platform

  • Subjects should never be required to provide personally identifiable information (PII) for their work to be approved
  • Set auto-approval time fairly - do not make your subjects wait longer than is absolutely necessary to find out whether they will be paid (≤7 days should be sufficient - if it is not, explain why in your IRB application) - and inform your subjects on the Preview page

  • Conditions for rejections should be clear and fair, as rejection of a subject's HIT can lower a subject's acceptance rate

  • Note that your sample may not be as diverse as you might have thought- research shows that MTurk workers are disproportionately liberal, single, urban, and college-educated

For more information regarding the use of Mechanical Turk, see Amazon's Mechanical Turk FAQs