Here is a quote from Greentarget’s DEI survey report (2023, at page 18):

“From April through August of 2022, Greentarget and Zeughauser Group conducted qualitative and quantitative research to inform the 10th version of this study. The firms conducted 10 in-depth interviews and distributed individual surveys to 200 executives … as well as more than 30 law firm chief marketing officers.”

As the two organizations did, sponsors of surveys not infrequently interleave and bolster their survey results with comments made by people they interview. Complementing qualitative research, which is based on what people say during interviews, with quantitative research, which is based on data gathered through an online questionnaire, makes for a powerful combination. In this portion, I will refer to one-on-one interviews, but much of what is said applies to multi-person focus groups.

I will note a few distinctions, however, between interviews and focus groups. Focus groups chill the potential candor of one-on-one interviews but they can generate a consensus response. You can even build a mini-data base if several people submit answers to a question. Ask them to write the question on a piece of paper along with their answer. Then, collect the papers and tally the responses. Peer pressure comes into play in focus groups as people listen to each other. Loudmouths dominate quiet, thoughtful types.

The option of interviews during a survey project raises a question: do you interview first, after the survey, or simultaneously? If you interview a sample from the population – such as one out of four randomly chosen partners – the insights can help you decide what key terms to use in questions, the questions themselves, and their order.

If you gather your survey data and then conduct interviews, insights from them can help you interpret the results you have found. (You will also likely realize all kinds of additional questions you should have asked on the survey or that questions were ambiguous.) If the interviews or focus groups follow the collection of data from a survey, you can dig deeper into interpretation of the findings. A conversation is hugely more flexible than text comment boxes. People are often more willing to talk than to put things in print and might feel safer knowing that you are aggregating comments from a number of interviews. You can assure individual interviewees of confidentiality; you can’t provide the same assurance to focus group members.

Not only do interviews open the sponsor to extensions of ideas and explanations of puzzling findings, they can also uncover critiques of the survey methodology or question set. Finally, if a law firm or vendor is sponsoring the survey, speaking with people creates marketing opportunities.

Interviews, however, are far more expensive, time consuming, and logistically challenging to schedule than an online survey. Even the effort to manually code text comments or detect themes in them doesn’t require as much time. Then someone has to aggregate or code the interview comments.

Additionally, interviews rely heavily on the questioning and interpretive skills of the interviewer. Then, if you have more than one person interviewing, it compounds the difficulty of putting the interpretations together. Finally, it is a challenge to pick representative people to interview. Whoever does that picking, unless it is random by last name or something, may influence the results.

Subjective comments are insightful and allow exploration of alternatives, etc., but nothing beats compiled numbers to persuade some people. To sit in front of people and collect numbers, then have to enter them into a spreadsheet is mind-numbing. Online surveys make it a cinch.