While developing a survey for a law firm about Work from Home policies (aka Return to Office), we arrived at a ranking question that included 16 potential advantages. That’s how many benefits to working from home we came up with (we developed close to that number of disadvantages, but no matter). The selections were so numerous that respondents might flag in their perusal of them, leading to primacy bias toward the first few selections being chosen more than they would have been if the selections were fewer.

What to do? Here are techniques that address the problem of generating a surfeit of selections. Whichever one you choose, or a combination, be sure to include an “Other” selection, because it is evident that a host of views are possible.

• Write tersely. If you are asking respondents to wade through many choices and evaluate them all, at least help them with brevity!

• Create two questions and split the selections. Perhaps a logical reason to divide the questions presents itself. But, no matter the basis for dividing them, the two questions won’t include every one of the selections. The obvious problem, of course, is that half the questions won’t be evaluated in light of the other half of the questions.

• Field two surveys, identical except for the question with so many choices. Each survey has a question with half the selections. No one will notice, and you can avoid burdening either group. This approach has the same downside as creating two questions on the same survey.

• After you get back a reasonable number of responses (perhaps 10-15), study the selections and drop the two that have the lowest frequencies or rankings. For example, say you have a long set of methods for controlling outside counsel costs. If the first ten people assign zero percent to “Choose small firms” and “Require firms to use our paralegals,” you might remove them from the selection list. You need to reassure yourself that the preliminary batch that you review was representative of the group you surveyed.

• Combine a few of the selections. Yes, the more granular the selections, the more insight you gain. Consider these two selections: “Outsource legal support from overseas” and “Outsource legal support from within the United States.” Leaving them separate tells you attitudes more precisely, but you could readily merge them. Still, broader selections lose precision.

• Put half of the selections on the questionnaire until you obtain a plausible number of responses, and then put the other half of the selections for the remaining invitees. After all, you are not trying to obtain a census, you are trying to understand the views of the lawyers. Once you have heard enough from them to reach a conclusion, gathering more data wastes everyone’s time.

• Drop a selection or two. If a selection cannot lead to constructive action, best not to raise expectations. On a compensation survey, when asking about changes respondents favor, don’t include “More restricted stock grants” if by corporate bylaws only two officers of the law department are eligible.

• Randomize the order of the selections. If you can’t prune, separate, or merge a long list of selections, at least have them show up for each respondent in random order. The primacy bias will be muted.

• Use conditional logic and re-ask a question with only the selections a person didn’t rank. To be honest, I’m not sure how to analyze the primary rankings with the conditional question rankings nor whether hosting software allows skip-logic questions based on varying selections chosen.