At first, it seems odd to think that quality surveys can be drastically better or worse than one another. They ask simple questions of opinion to people who will answer them, hopefully honestly. If the survey ends up producing the data one wants is almost entirely out of their control.
On some level this is true. They’re assessments which cannot be controlled. Although that’s part of the difference between a good and bad survey, does it influence the participants? Ultimately what makes a survey good is how direct, honest, and clear it is. Beyond the survey itself what is even more important is how it’s interpreted and reported.
Moving back to what makes a survey itself good, it’s important to elaborate on that a bit. Questions are the heart of all surveys, and they range drastically in quality. Questions can lead to answers, make assumptions, and be unclear. The difference between a good and a bad question can be very subtle.
For example, to know how someone feels about a product, there are a lot of questions to ask. How do they feel about the delivery, the quality, the longevity, etc. I can ask if someone liked the product, but that’s unclear. Asking specific and distinct questions is essential to having strong data. Did the product arrive on time? Do you use it? How often?
Moving away from the questions, another big difference is in how the survey is executed and distributed. If there’s no or poor planning, it’s not uncommon to have too small a sample. A small sample means the data cannot be generalized and is very limited in use. Another potential error is in influencing the participants. Digital surveys are great at getting around this but bias is still a real concern.
Even when the participant is finished, where is that data going? Is it secure? Are all participants data going to the same place? It’s questions like these that need to be understood and planned for. If not, even with a lot of participants, the data becomes useless.
Now what about after all of that? Here’s where the importance of analysis and reporting comes in. Data must be analyzed properly. This means having statistical knowledge, adequate resources, and ample time. Rushing the process can skew results and lead to later issues. There are workarounds, work can be exported, questions can be removed, but it must all be deliberate and clear.
Finally the reporting is possibly the single most important difference between a good and bad survey. If the data is not able to be understood, it has no value to anyone besides the creator. Here is where graphs, tables, and clear design are everyone’s best friend. Of course it can be frustrating that the display is so important, but it is. Data is collected to be used and to make a change. If it can’t do that, there was never a point in the first place.
These are the things that make or break a quantitative survey. Each individually can feel small and easy to overlook, but they are essential. There’s a lot of doubt today about subjective surveys and methods. Surveys themselves are a great tool, but it’s people’s misuse of them that creates doubt. Don’t make these same mistakes, make a good survey.
