Human Research Protection Program

The Brandeis University preference and default is that investigators use Qualtrics when conducting online surveys. Qualtrics is a cloud-based survey tool for which Brandeis University has a license for all faculty, staff and students. Qualtrics has been vetted by Brandeis University’s information technology services department and is considered a secure system appropriate for all data, including sensitive data and protected health information. 

Limit your questions to those that are truly relevant to your research question. Eliminating questions that may be interesting but aren't really necessary to answer your research question is important not only to keep your survey as short as possible (which may reduce dropouts and simplify your data analysis), but for issues of privacy and confidentiality.

Keep in mind the more questions you ask, the easier it will be to identify subjects and link them to their answers. This includes questions that may seem innocuous — even demographic questions.

For example: You survey the student body and ask your subjects to provide you with their major, sex and ethnicity. Your response rate is good: 70%. Still, only one student identified themselves as a language and linguistics major, male and Hispanic/Latino.

Not all surveys are anonymous. The terms "anonymous" and "confidential" are often confused, and many investigators promise their subjects that a survey will be anonymous when, in actuality, it is not.

Anonymity can be promised only when it is impossible for anyone (including the principal investigator) to link the survey answers to any information that could potentially be used to identify or trace a specific subject. As discussed above, demographic variables — or even the answer to one simple question on a survey — can potentially be used to link the entire completed survey to a single individual.

Some things to keep in mind: Use as large a sample, as diverse a sample, and as few demographic questions as you can. And think carefully about all potential identifiers; one identifier that is often overlooked when conducting surveys online, for example, is computer IP addresses.

Note that Qualtrics' default is to collect IP addresses; however, you can remove them by enabling "anonymize responses" in the survey options.

Confidentiality, on the other hand, refers to the expectation that a subject's information will not be divulged to others without his/her permission, and that you, as the investigator, promise to do all you can to keep the subject's information from being discovered. Because anonymity can be difficult to achieve, most research conducted is considered confidential.

Informed consent is the process by which you, the investigator, explain your research to your prospective subjects so that they understand the research, the possible consequences of participating and that their (ongoing) participation is completely voluntary.

When conducting a survey in person, the subject is usually asked to sign a consent form. When conducting a survey online, informed consent is usually obtained electronically.

Note that Qualtrics offers the ability to collect actual signatures using a mouse (on a computer) or finger (on a phone or tablet).

E-signatures (typing rather than signing) are permissible provided the subject's identity is verifiable. 

Online surveys often begin with a screen display with the information that would be contained in a traditional consent form, and a checkbox that indicates the subject's acknowledgment that s/he has read the information provided and consents to participate.

For information on how to construct an informed consent form, see the Informed Consent page of the HRPP website.

When conducting research, there is always a risk of negative effects, such as psychological distress from being asked questions they aren't comfortable with, and the potential for a breech of confidentiality and its possible ramifications (such as legal or social). Prospective subjects should always be warned about these possible harms during the informed-consent process and given the opportunity to decline to participate in the research or skip questions they don't want to answer (be sure this is an option with your survey software).

Consider the risks to subjects as you develop your survey questions. Are you asking personal or sensitive questions and, if so, are they truly necessary to answer your research question? Be mindful of the fact that when conducting research online, you cannot gauge the reaction of your subjects (such as if they find the questions stressful).

If your questions have the potential for harm — especially if more than minimal (where the probability and magnitude of harm or discomfort is greater that that ordinarily encountered in daily life or psychological examinations or tests), include a debriefing at the end of the survey with information on whom they can contact for help, if they wish (such as counseling services or online support groups).

You may choose to compensate subjects for the time and effort they put into completing your survey. When compensating your subjects, there are a few things to be aware of:

• First, is how much, and whether you pay each subject the same amount or use a raffle, where each subject has an equal chance to win something. Inherent in this is what the compensation will be — cash, a gift certificate or an object such as an iPad. 
• Second, is whether they will be compensated only if they complete the survey, or receive the compensation even if they withdraw before they complete it. If compensation is not contingent on completion of the survey you must include a way for the subjects to collect their compensation. For example, you may want to include a button that will take them to the end of the survey where they can pick up their gift certificate or provide their contact information.
• Third, if collecting their contact information for compensation, always have the survey bring the subject to a new page, separate from the survey; this will disconnect the survey data from their personal information (unless the survey itself requires your subjects to identify themselves in order for the data to be accepted).

One risk you can never avoid with surveys that are not anonymous is a breach of confidentiality. Two things you should consider:

1. Limit who has access to the data.

2. Use the highest level of security possible; be sure your server is secure and your data encrypted, for example. Brandeis' policy is to use box.com whenever possible.

If you have any questions regarding data security, contact the Brandeis University security office.

Amazon Mechanical Turk, known as MTurk, is becoming a popular platform for conducting research; it's thought to be cheap, easy, anonymous and to reach a large and diverse population. Unfortunately, however, MTurk does not come without its problems.

Before using MTurk to conduct your survey, be sure to read Amazon's MTurk privacy policy. Note that Amazon.com has stated that the MTurk platform is not meant to support subject anonymity. MTurk worker IDs are linked to workers' Amazon.com profiles, often allowing one to see such things as the subject's buying habits, wish lists, video tastes, product reviews and sometimes names and photos. In addition, MTurk automatically collects subjects' IP addresses, which can be potentially identifiable, depending on context. 

For this reason, MTurk should only be used as a recruitment tool with a survey link embedded within the MTurk task that redirects your subjects to complete the survey at an external site such as Qualtrics. Have Qualtrics assign your subjects unique completion codes, which can then be provided in the MTurk task in order for the subjects to be paid through MTurk. 

If you must collect your subjects' worker IDs, these should be de-identified as soon as is feasible. This process must be explained in your IRB protocol and mentioned in your informed consent page.

Some additional things to consider when using MTurk as a recruitment tool:

• Consider the appropriateness of your human intelligence task (HIT) title. Keep in mind this is how you are recruiting subjects.

• Include any screening questions on the Preview page or use a qualification test to screen prospective subjects — or compensate for completing a screening survey whether or not the subject qualifies for your research project.

• Include a description of the research on the Preview page. What will subjects be asked to do and how long will it take? Be sure to be reasonable — overestimate rather than underestimate.

• Include basic consent form information on the Preview page.

• Compensate fairly — pay should be equivalent to what your subjects would be paid if you were using any other recruitment platform.

• Subjects should never be required to provide personally identifiable information for their work to be approved.

• Set auto-approval time fairly; do not make your subjects wait longer than is absolutely necessary to find out whether they will be paid. Fewer than seven days should be sufficient. If it is not, explain why in your IRB protocol, and inform your subjects on the Preview page.

• Conditions for rejections should be clear and fair, as rejection of a subject's HIT can lower a subject's acceptance rate.

• Note that your sample may not be as diverse as you might have thought. Research shows that MTurk workers are disproportionately liberal, single, urban and college-educated.

For more information regarding the use of MTurk, see Amazon's MTurk FAQs.