Toward the end of most academic articles, the authors reflect on the research that spawned the article and speculate on how they might have done better. They also predict how they would like to extend that research in various ways. It is a self-critical postmortem, and that practice suggests what survey sponsors should also do.

In the four topics below, I suggest general areas of self-analysis and reflection. The animating idea is objectivity improvement, and tough love.

What would we do better next time? Everyone can wish for more survey participants, but here is the place to suggest plausible ways to accomplish that. Or you would ask another question or two (or drop ones where no mystery remains). Describe how you might have improved questions, like asking for the number of locations of law firms only where there is a permanent partner and associate and therefore it’s not a mail-drop location. Looking back, you wish you had improved validity checks for answers or qualifying questions. You might in retrospect feel you should have focused on a size range and eliminated smaller and larger participants (or treated them differently in the report).

What would we not do? Candidates for this category could be questions that resulted in ambiguous answers, answers that could support multiple interpretations. Going forward and repeating the survey, you might resolve to drop questions that require research by respondents, because those lower participation rates. Or you might decide not to deal with currency conversions or to change how you instruct respondents to do that. It may be you realize at the end that mailing out surveys cost too much for the few responses it generated.

How might we introduce new ideas or further findings? Perhaps you ask for age, gender, or another piece of supplementary data that strengthens your findings. You might ask for data from the previous year plus the current year and thereby create instant trend data – assuming the passage of a year allows enough time to make an appreciable difference. You might ask an open-ended text question instead of trying to capture the complexity of life with multiple-choice questions.

How might we extend our research? Academics always want to explore more in their field of professional interest. Each article builds toward the next step. Ideally, those who conduct surveys in the legal industry also dream of fashioning better surveys, asking more insightful questions, and exploring more provocative areas of thought.

With legal industry surveys, however, a factor inhibiting such disclosure might be that law firms and legal vendors do not want to disclose to competitors where they foresee inquiries might go. Why tell your business opponents good ideas about trends you plan to research in the next survey or ideas for more sophisticated machine learning interpretation? Another discouragement to laying out future directions might be that your law firm or company does not plan to conduct the survey again and therefore whatever lessons you draw from the survey you have completed will reap no benefits for a subsequent version.