Many designers of surveys like questions that ask respondents to indicate their opinion by choosing from a scale. Likert scale questions (named after their creator, American social scientist Rensis Likert) are popular because they are one of the most reliable ways to measure opinions, perceptions, and behaviors. Such topics can’t be understood with one question. Instead, you collect data on multiple indicators — questions that help you understand the concept you’re trying to measure.

Take client satisfaction with the performance of a legal department. It’s a sense clients develop where several factors come into play: perceived quality, creativity, knowledge of the law, speed of response, accessibility, and so on. Each of these subjective views becomes an indicator to help find out if how corporate executives feel about the legal team.

The most common scales offer either five or seven response options. An odd numbered scale also lends itself to two negative choices and two positive choices. With an odd number, respondents can settle on a neutral or no-view middle option, a three or a five respectively. From Pollfish, “the choices [should be] evenly distributed in weight and symmetry across the scale and ranging from one end of a spectrum to the other. “How often do you wish the Law Department would retain outside counsel?” Much more; Moderately more; About the same; Moderately less; Much less. Others advocate an even number of response choices. Why? Because it forces people to choose a side, making it easier to collapse responses into two categories (positive vs. negative experience). From Typeform, “The jury is still out on whether that is necessary or even desirable.”

Note that positive and negative indicators are balanced. A more reliable survey design puts a positively worded statement before or after a negatively worded counterpart. So, if you have the statement to be rated, “The variety of employee benefits confuses me,” then you should also have a reverse statement like, “I appreciate the wide selection of employee benefits.” People who agree with the positive statements should disagree with the negative ones, and vice versa. That’s how you can be confident that your questions are clear, and the responses are reliable. (It also lengthens the survey.)

Choose options that are simple and unambiguous. Among the most common: Agree—Disagree, Helpful—Not Helpful, Excellent—Poor, Satisfied—Dissatisfied, Always—Never. Make sure the differences between the categories are valid and useful. Let’s say you want to measure how often an associated asks for assistance from a paralegal. If you choose Never—Seldom—Sometimes—Often—Always, how do you quantify the difference between seldom and sometimes? If a scale is potentially ambiguous, either explain the meanings in your introduction or change the scale. Don’t use ‘Sometimes’ when you really mean ‘Once a week.’

Matrix questions allow Likert scale questions to ask about several ideas at once. This can be a good option when the survey would otherwise be repetitive, measuring on the same scale for many related indicators.

Likert scale questions are vulnerable to respondent biases. For example, even when you tell respondents their responses will remain anonymous, many people still try to give socially acceptable answers rather than being honest. Social desirability bias can be minimized with an even-number scale, which prevents people from sitting on the fence.

Likert scale questions are also susceptible to central tendency bias: people avoid choosing the most extreme responses such as Very Helpful or Strongly Disagree. You can reduce the effect of this bias with clear definitions, such as “’Very Helpful’ means you got what you needed from the Legal Department.”

Averages, medians, frequencies, and standard deviations describe the data from a Likert scale question. In addition to these statistics, you might choose to aggregate the answers into what is known as a Likert plot. On a recent project of mine, one question asked about certain views of the law firm’s partners. Here is a Likert plot that shows the distribution on four of the questions (the indicator names are made up).

Likert plot

As an example of how to interpret the graphic, the “Morale” indicator at the top had 35% negative views, 19% neutral, and 46% positive. The finer breakdown within negative and positive shows up in the two colors. This calculation tells you a net supporter score. You subtract the total of the two negative indicators, for “Morale” it is 35, from the total of the two positive indicators (49). Voila, a NET supporter score of 14. Going one step further, you can average all the net supporter scores into a single, grand summary of the opinion, perception, or behavior that you want to learn about.