Decide When it’s Time to End Data Collection:
All survey sponsors crave more participants. More participants mean more data, which signals interest in the topic and fertilizes robust analytics. But all good surveys must come to an end, so when do you decide to stop accepting new responses? For a one-shot survey, the curtain has to come down; if you plan to return to the survey next year, you will be more open to keeping the survey open. I have encountered these nine stop signs.
Reached the Population: If your law department is surveying its lawyers, and 23 out of 24 of them have submitted their surveys, that marks the end. You’ve covered all there are
Deadline: You might need to publish the report by a conference date or by the time of an important presentation. To meet deadlines, you ought to allow enough time to polish the report or presentation slides. As another example of a deadline, if a major court decision is handed down, a survey that taps how law firms or law departments might respond to it needs to get out and done quickly.
Commitment: You might have made promises of delivery time. “Complete our survey and receive the report in two weeks!” A quick turnaround helps lure participants – but they deserve to have you deliver on your assurances.
Goal: You may have decided at the start of the project that when you reach a pre-determined number of participants, that is sufficient. A round number like 100 may be the target. No one I have encountered in the law firm or law department world figures out how many respondents they need to achieve a respectable statistical margin of error but they might have a gut feel for what their target market will deem to be a credible total.
Balance: You have enough participants in each strata (even before weighting). Your outreach to law firms has collected the views of 50 partners, 100 associates, and 25 members of management committees. That’s sufficient and balanced.
Dried Up: Two new responses trickled in last week and only one this week – time to turn off the spigot. No flow and no ongoing efforts that are likely to elicit a flow.
Money: You might be forced to terminate because you have run out of funding, as for example, if you have proposed to conduct a survey on a fixed fee, and the budget is trickling away. No survey project has unlimited resources.
Aged data: Survey Information beyond its warranty date, so to speak, may go stale, as for example if a compensation survey begins late in the calendar year. It is too late for respondents to act on the report’s findings. If the timing is off, recognize the futility of waiting for what won’t come.
Seasonality: Fewer people will have the time or inclination to take a survey in December and in August. Why keep it open during a dry spell?