Ask About Sensitive Topics Indirectly

Some topics are so likely to trigger discomfort among respondents, and thus partial participation or honesty, that you don’t want to ask respondents to answer your question directly for themselves. Say you are interested in whether your firm or law department should create co-ed bathrooms. To ask in the questionnaire point blank, “Do you favor co-ed bathrooms?” could cause many participants to pause, feel uncomfortable, and not answer. Worse, they might abandon the survey at that point.

If you are thinking that the reluctance springs from the rigidity of “Yes/No” questions, where the respondent yearns to give context and nuance to their answer, e.g., “Only a few bathrooms would be designated co-ed” or “Only for adults with children under 10”, you would have a fair point. But a controversial question also invites social desirability bias, where respondents answer in the way they believe the sponsor wants them to react. For some people, the topic is simply too personal, too upsetting, too controversial – an invasion of their private life.

But if instead you ask, “On a scale of 1-7 (where 1 is adamant opposition and 7 is strong support), how do you believe your colleagues would feel about co-ed bathrooms?”, you will likely receive significantly more answers. With this oblique style of question nobody is pressured to reveal their deeply felt personal beliefs. Nevertheless, because people often project on others how they feel about a topic, you have an indirect measure of individual attitudes. Most people believe they are in the mainstream of opinion, so if they give that question a 2, you can infer that it is likely they do not favor co-ed bathrooms.

A second, more sophisticated survey technique is available. A “list experiment” is an oddly-named questionnaire device that can mitigate respondent social desirability bias when eliciting information about sensitive topics. With a large enough number of respondents, list experiments can be used to estimate the proportion of people for whom a sensitive statement is true. The survey presents respondents with a list of items, experiences, or statements and asks them to select how many items in the list they agree with or pertain to them. Here’s the trick. The invitees are randomly divided into two groups: the Direct Response Group and the Veiled Response Group. All that is required is two URLs or a conditional gating question. While the Direct Response Group is presented with a list of neutral and non-sensitive items, the Veiled Response Group is presented with an identical list plus the sensitive item somewhere in it. With enough responses, survey researchers can estimate the proportion of people to whom the sensitive item pertains (or which they agree with) by subtracting the average response of the Direct Response Group from the average response of the Veiled Response Group.

A third method to obtain input on contentious or upsetting topics relies on a rating or ranking question. You can ask respondents to rate the following issues by how important they are to be addressed and include in the list “Establishing co-ed bathrooms.” Those more inclined to favor co-ed bathrooms will probably rank that issue more highly than those who disagree with creating such bathrooms. This is a roundabout method, which depends on inferential conclusions, but it gives insight into how respondents feel about a touchy issue that can’t be tackled head on.

In summary, at least three methods exist to elicit useable responses on tricky topics: ask for collective views rather than individual views, split your response base and utilize a list experiment, or ask for evaluations of multiple issues, one of which includes the sensitive topic. If your questionnaire includes more than one of these methods, the combination of answers solidifies your conclusions.