Survey Says (Part 1 of 5): Using Surveys in Education
If you’ve been in education for more than a few years, then you’ve probably lost track of the number of surveys you’ve had to participate in, either as a creator or respondent. Often, when we take surveys, our minds reach a point of thinking, “I’d rather be doing anything but this.”
There’s an Econ 101 expression that says, “There’s no such thing as a free lunch.” I won’t go into whether that’s true or not in all cases, but it certainly applies to the use of surveys in education. Every time you ask a stakeholder to do something for you that requires money or time, you are incurring a cost, and so is your stakeholder. No one has unlimited money or patience. You have to be intentional about what you ask of your community and how often you ask them for something.
As a thought exercise, let’s consider a 5-item survey you want to email to your students’ parents. Assuming this survey takes 3 minutes per person, on average, and you expect 150 parents to respond, that’s 450 minutes of parent time taken up by this survey. That’s 9 hours of people looking at their screens and giving you information. That may not sound like a lot, but you can’t ask for those same nine hours for something else. And you won’t have many more opportunities to ask for more time before stakeholders stop responding. Are you sure that this survey is worth this risk?
In this five-part series (this is part 1), I am going to share with you some of my lessons learned over a career of creating surveys that will help you make these decisions. I will cover topics related to planning, creating, and analyzing your surveys. For now, read through the list of questions below and think about a recent survey you’ve administered or a new one you have in mind. While this list doesn’t cover everything you should consider before launching a new survey, it is fairly representative of the types of questions you should ask yourself before you start asking other people questions.
- Why is this survey necessary?
- Is this survey worth the time it will take your respondents to complete it?
- What questions do you have that the survey data will answer?
- How do these questions align to your strategic plan?
- How will you respond to the data you collect?
- Will the survey items generate data that look like the metrics you need?
- How will you administer the survey?
- How many respondents do you need to be able to trust the results?
- How will you recruit that many people to take the survey?
- How much time do you expect each person to spend taking the survey?
- What will you do if you don’t get enough people to respond?
- Who is going to analyze the data?
- What types of measures (e.g., averages, counts, etc. …) do you plan to use?
- What types of visuals will you use to communicate results?
Practitioner’s Corner: Estimating Survey Time
If you’ve read the article above, then you already have a good idea of how to estimate the amount of time it should take a stakeholder to complete your survey. Once the survey has been created, this is much easier. You simply have a few people take the survey and track the time. Then multiply that number by the number of people you expect to participate. Before your survey is created, though, it’s all about estimation. It takes the average person about a minute to read 250 words, which is more than you should write for instructions. But you can use one minute as a conservative estimate for instruction reading time. From there, plan conservatively for about 20 seconds per question (3 items per minute). If you’ve got a 15-question survey, this should take your stakeholder roughly 6 minutes to complete (including instructions). If you’re hoping to get 250 responses, that’s 1,500 minutes. That’s 25 hours of stakeholder time you’re requesting. This type of exercise should be a starting point for every survey. Stakeholders can tell when you don’t value their time. If you send out surveys, even brief ones, without considering what you’re asking from people, you will end up paying for it in the long run.
