I urge survey designers to reconsider – – polite talk for shun – – Yes/No questions because most of life scoffs at such a solar flare versus Stygian darkness view of anything; the messiness of life almost always demands more complex, nuanced answers. Sure, “Were you born in Texas?” admits to a binary answer, but just as surely, asking internal clients, “Are you pleased with the service you receive from the law department?” does not.

According to Zapier’s blog “The simplest survey question—and the only question you’ll usually use in a poll—is a Yes/No question. You’ll ask a question, then have two options: Yes and No.” SurveyCrest explains that if only two choices are permitted, such as “Yes” or “No”, the question is known by survey aficionados as a dichotomous nominal question.

Proponents of Yes/No questions want to drive to the ground truth: make the respondent take a position: “Do you have the training you need to redline contracts proficiently?” Don’t let them waffle! No “on the one hand, but on the other hand…”. Furthermore, yes/no questions translate into handsome pie charts (another abomination). At first blush, they seem to save respondents time – just a quick yea or nay. But, if the respondents have to weigh multiple factors and figure out their preponderant view, the clock ticks on.

It exacerbates the mistake of asking for a binary yes/no if you combine two or more of them with a disjunction (expressing a choice between two mutually exclusive possibilities). And this criticism extends to how you write selection items for multiple-choice questions or priority questions. If you ask, “Please check your strongest objection to required return to work:” and one item from the selection group reads, “Can’t find flexible day care or schools close unexpectedly” you are left unsure which part (or both parts) justified the answer.

What’s wrong with an online survey question that asks, “Do you like the law department’s speed of response or knowledge of the law?”? One wrong is that when it comes time to compile your report, for those who answer “Yes,” you don’t know which side of the “or” they like, or perhaps it is both sides. For the “No’s”, are they critics of speed, knowledge, or both? What if the respondent likes the speed of the department, but mistrusts the in-house lawyers’ knowledge of the law? How should they answer the question as posed?

Equally problematic would be a similar question that exchanges “and” for “or.” “Do you like the law department’s speed of response and knowledge of the law?” At least a “Yes” to this covers both attributes, while a “No” to this question suggests perceptions of shortcomings on both alacrity and acumen but it leaves a logical gap – what if only one attribute is disliked?

Three corollary observations: A box for text comments won’t mollify anyone. The proffered explanation of a viewpoint has to be parsed (coded) by an analyst if it is to contribute to the survey analysis. Furthermore, with both flavors of “conjunction questions”, our brain takes more cycles to sort out, and time elapsed risks survey fatigue.

Third, if you (or your client) insist on those clumsy, Procrustean questions, at least break the conjunctive question into two. Up or down may work for simple polls – “Did you vote for X?” at an exit interview, but the stark choice rarely deals well with the complex world. Far better, recast the separate questions to permit a response on a scale, such as “On a scale of 1 to 7, with one being terrible and seven being excellent, how well do you rate the law department’s speed.” The next question does the same for knowledge of the law.

I will add another criticism of the questions we started with. What if “like” misses the verb the respondent would choose: “respect”, “appreciate”, “put up with.”? Nor is there any adverb to modulate or qualify their opinion such as “for the most part,” or “mostly.” And while I am carping, the question does not set a backward time frame for the evaluation.