Many surveys seek ratings of a feature, a policy, an action, or some topic. Respondents typically choose from a limited scale, such as a Likert type question, to provide their evaluative rating. Analysis of those responses provides insights and powers many online surveys by law firms, law departments or legal vendors. If a compensation survey asks lawyers to assess their satisfaction with vacation time, pension plans, medical leave, CLE funding, and other benefits, that’s fine as far as it goes. However, having respondents rate each item doesn’t indicate how relatively important they view it compared to the other items.

For that reason, sponsors should follow the rating question with a corresponding, similarly structured question that asks respondents to indicate the relative importance of whatever has been rated. The relative importance question could ask respondents to allocate 100 points among all the choices according to which they felt were most salient, or it could ask for them to pick the top three or four items (other possibilities exist to prioritize the items). Or it could ask for a forced ranking, such as one-third high importance, one-third medium importance, and one-third low.

The survey questionnaire might be designed as a table or matrix so that in one column it holds the evaluation rating and in the next column the relative importance score.

Whichever way the survey proceeds, the analyst can prepare what is called a “gap analysis”. A gap analysis shows where evaluation ratings differ most from importance scores. To give a whimsical analysis, if a law firm surveys its clients about the performance of the firm, the firm might get high marks on penmanship, but the importance of penmanship would be in the basement. What you look for in a gap analysis are attributes or characteristics that are deemed considerably more important than their ratings. The effort should be directed to narrowing those gaps. Conversely, high scores on unimportant attributes suggest over-investment in activities relative to their value. It hardly matters to excel at trivia.