Borrowing from the saying, “Data leads to information, which creates knowledge, and cumulates in wisdom,” we can analogize to the ladder of value and sophistication that survey reports can offer.

Response data presented in the raw, as numbers, is the most fundamental output in a survey report. You show tables or graphs that aggregate and represent the data from the survey respondents but do little more than regurgitate the numbers and explain how to read them. “Four out of five law departments have no systematic knowledge management initiative.” That is a raw number; count the departments who say they have the initiative and divide by the total number of departments that answered the question. Raw data can be massaged mathematically and statistically, such as by being imputed or normalized, but the bedrock is a numeric statement: “35% of e-discovery analysts have no academic background with software.”

Findings represent a distillation from raw data. They are closer to information than a data dump. Findings are generally in the form of text summaries of what a reader of the report could glean on their own from tables and charts. A finding might be that the median total compensation of general counsel was $750,000, which is 40% more than the median for associate general counsel. A finding gives context to raw numbers and links to a larger understanding, but unlike conclusions, it is grounded in and limited by the reported numbers.

Conclusions is a term for what the survey sponsor derives from the findings. For example, a finding might be that increasing years as a chief operating officer correlates to increasing total compensation, but a conclusion might be that while this is true, it plateaus around 10 years and based on regression models is not predictive. The conclusion might add the obvious, you can’t speed up years of experience. Quotations from subject matter experts or others usually have a whiff of conclusion about them as they put the finding into a real-world framework. Conclusions represent thought about findings.

Recommendations are based on the three less sophisticated outputs: raw data, pulled together as findings, and extended by conclusions. Recommendations urge the readers of the report to act. If the collected survey data, boiled down to findings that yield broader conclusions are persuasive, then readers want to know what they should do going forward. Recommended actions represent the epitome of what a legal-industry survey should strive for. From having read hundreds of survey reports, I would estimate that fewer than 10% make explicit recommendations. Here is a screenshot from the report by World Services Group of its Generation Now Survey (pg. 9). At the bottom of the page, WSG has a recommended “Action.”

These four levels of sophistication depend greatly on the quality of the survey’s methodology. In fact, to be honest, everything about a survey revolves around the carefulness and mathematical scrupulousness of the sponsor in all aspects of their survey project.