On a survey of law firm lawyers, what is a weakness of the following question? “On a scale of 1 to 7, with 1 being terrible and 7 being excellent, how do you view the firm’s IT department responsiveness?” The blunder is that it places no temporal limit on the respondent’s retrospection. Some respondents might have long memories, or a dramatic example of the far past remains salient – “Remember that time six years ago, the law department dropped the ball completely.” You can’t know how far back they might recall. Much better to start the question with “Thinking back over the past 12 months, on a scale of ….” Now you have improved your precision and standardized the answers with a bounded time frame.

Even when you ask respondents to project ahead, frame a question as “Looking forward over the coming 18 months, what are the most important diversity goals the firm should pursue?” This time frame covers the future, but at least you have specified what portion of the future.

A time frame adds precision and credibility to the answers you collect. It delineates a period of days, weeks, or months. The period chosen might be meaningful, such as while the law department was short-staffed, consumed by a merger, or relocating; or it might be an arbitrary boundary such as “during 2021.”

The time frame might frustrate a respondent, so you might appease them with a text box. The respondent may be bursting to bring to your attention the shortness of the time frame, a crucial event or perception just outside the limit, or the inappropriateness of the given time frame.

What I have never seen is the flexibility to let the respondent pick the timeframe that they believe is meaningful. They could answer based on the previous six months, year, two years or whatever. Their choices of time periods (and justifications for it) might inform you of history you did not appreciate at the start. However, I fear that such a question would throw doubt into the solidity of conclusions you draw. Unless you have a plenitude of answers to each optional time you might not be able to do much with analysis.

Perhaps it does not matter to you to control the length of look back. You still collect a rating, and an open time frame gives flexibility to respondents. Or you may feel that any period designated by you is arbitrary and could err in either direction (too short or two long). It’s a tradeoff, but I strongly favor the known period over a shapeless past.

One more point about time frames in questions (or in the introductory text for a series of questions – “In the following six questions, answer for 2022.”): A question set up in a table can gather feedback on two periods in the past. “How was administrative support in 2019 (before Covid)?” would go in column one, and “How was administrative support in 2021 (during Covid)?” would nestle next to it; together, the responses show a shift over a three-year stretch. The benefit of asking double timeframe is that you have effortlessly created the potential for a trend analysis. The disadvantage of paired questions is that the taxes the brain of the lawyer or paralegal who is being asked to mark off attitudes across time.

Related to this point is setting a point in time for a question’s answer: “What was your base salary on January 1, 2022?” leaves more reliably similar answers than “What is your base salary?”. With the latter, a person might just have been promoted, gotten a raise, or moved to part-time status.