Here are several clues that suggest a respondent is merely trying to get the report but doesn’t deserve to because their answers were too weak or questionable. Clues can point you to suspects, but it is a good practice to write to them and inquire calmly and objectively about their dubious response. Perhaps they were rushed, misunderstood instructions, or were interrupted while replying. Who knows? Or perhaps they lack genuine bona fides.

Takes little time: A Stack Overflow survey of developers conducted in 2022 considered responses to be “qualified” for analytical purposes based on time spent on the full, completed survey; approximately 53 responses were dropped because respondents spent less than three minutes. If your hosting software reports how long someone spent on the survey, you can set a minimum period of time because you know how many questions there are and you know the median amount of time taken by those who completed the questions. Too quick a brush over the survey damns a response.

Enters straight-line answers: I pay attention to answers where questions ask for percentage estimates or rankings. If someone enters a uniform 5% for all time estimates, for example, a string of 3’s on a Likert-scale question, or ranks all seven factors as “highly important,” I wonder whether they actually gave thought to the responses. It seems more likely they simply plugged in the simplest and quickest answers.

Leaves a high proportion of missing answers: If a survey response lacks answers to many questions, especially those that ask for substantive values other than demographic information, it raises suspicions. Someone who intends to fake their response and do the minimal amount of work to fly under the radar is not genuinely concerned about the issue but may want others to do the hard work. Those pretenders draft in behind and claim the unearned report.

Completes only the first few questions: Another pattern suggests a respondent lacks a credible desire to participate. A person who fills in the first few, easy questions about name, title, and state, for example, and then leaves blanks for the meaty questions about the survey topic has demonstrated no commitment. They do not deserve the report you prepare.

Enters no text comments: If your survey of law firms or law departments asks for open-ended text responses to provocative questions (e.g., “What issues do you foresee for your position in the next year or two?” or “Please described how return to work has fared in your department?” you wonder about the engagement of the person who enters nothing. Even more so the omission stands out when almost everyone else wrote something.

Enters “mistakes” or non-sensical answers: If someone enters $1 million dollars for their bonus, or if they answer that their most important concern with new lease expansions is carbon emissions, you could be pardoned for questioning their seriousness. True, some respondents read scales incorrectly, so they rank high what most others rank low, and true it is that unusual concerns surface, but a pattern of answers that make little sense or contradict puts doubt in your mind.

Provides a personal email address: My antennae alert me to the person ostensibly from a law firm or law department who enters not their business email address but a Gmail account or other personal address. You have one less fact to go on if you are debating whether the response reflects actual figures and true beliefs. They may, of course, simply want the report to stay private and personal to them, to have it outside the network of their employer; still, it vouches for authenticity to identify yourself and your firm or company.

In the end, to disqualify a response takes careful judgment. You may have clues from the above, but it is the composite or pattern of inputs that triggers a suspension. And, as I mention at the start, best to ask the person about their problematic response, and weight that as the clearest decision-making tool. Notably, if they don’t reply to your email, the case is closed.