As you carry out surveys, you begin to realize that many decisions you make cut both ways, so to speak. You gain because you take one fork, but you lose something too. I consider these bi-valent decisions to be under the rubric of trade-offs. The more aware you are of what you give up to gain something, the sharper will be your survey projects. Thought of most broadly, every survey project creates opportunity costs, a manifestation of a trade-off. In alphabetical order below, I have compiled several other survey trade-offs I have recognized.

Anonymity: If you require respondents to say who they are or who they represent, you may undermine the candor that you seek. You may end up with fewer surveys in your data set. Some people are unwilling to put their name to their response and therefore bag the whole thing. But anonymity blocks the sponsors from a variety of useful steps.

Binary questions: Simple, yes. Seemingly clear in their interpretation, yes. But every “Yes/No” or “Like/Dislike” binary question you include on your questionnaire runs the risk of forcing respondents into black-and-white choices that don’t reflect their nuanced views.

Facts known to sponsor: You don’t want to waste respondents’ time or bog them down with supplying simple facts, yet you ask them what you already can know. You want from them answers you cannot generate yourself, not facts available to you. The ideal supplemental data consists of contextual information that the respondent does not know off the top of the head, such as the rating of a lawyer in a law department in terms of succession planning, the number of years a law department has retained a law firm, or the market capitalization of a publicly traded company.

Length of survey: Participation rates are in a trade-off with the perceived length of the survey as attrition will take a toll on a long questionnaire. The best way to shorten the time of a survey is to delete questions that add insufficient value. Avoid the lure of letting people think, “Oh, add it as the answers might be interesting….”.

Long introductory text: At the beginning of the questionnaire, an enticing preview of what follows will help hold casual inspectors of the survey and lessen attrition thereafter. If people know where they are headed and the time it will take, they are more grappled. Still, drone on an on with “here’s what’s coming” and they might never proceed on.

Multiple choice vs free text: The balance of selections versus text boxes tilts toward more text answers, as the findings will be more nuanced, even totally unexpected, but yet the labor required to sort out the comments and categorize them can be daunting.

“Other” selections: The more often respondents check “Other”, the more work an analyst has to do with clean up. We might have sidestepped vexation by writing, “Please select the title that most accurately corresponds to your title” and thereby eliminated the invitation to irritation (but losing detail, so a trade-off).

Prioritizing questions: They demand much more thought from a conscientious respondent than does the simpler “Check the most important” or even “Check the two most important.” They require the person to evaluate the entire list of items relative to each other in stack order (ranking) or against a standard scale (ratings). These questions are accordingly much more cognitively demanding. As a result, the analyst can be less sure of the answers being diligent and thoughtful. Worse, they might fatigue or frustrate respondents to the point that they drop out. On the plus side, however, they allow more insightful findings, and thus present a trade-off.

Required questions: Sponsors of surveys often have an understandable desire to make many questions mandatory. Respondents, however, favor few or no required questions. Hence the inherent trade-off: sponsors crave information, responders dislike coercion.

Time periods: Perhaps it does not matter to control the length of look back in a question: “How often have you been dissatisfied with HR?”. You still collect a rating, and an open time frame gives flexibility to respondents. Or you may feel that any period designated by you is arbitrary and could err in either direction (too short or too long). It’s a trade-off, but I strongly favor the known period (“over the past 18 months”) over a shapeless past.

Rather than cataloging more instances, let’s summarize. The more sensitive you become to trade-offs inherent in survey projects, the more you realize their ubiquity. Every decision you make raises trade-offs, whether you are aware of them or not. The more you are sensitive to trade-offs, they better you can weigh the various considerations.