Questionnaires are a great way to find out the opinions, intentions, behaviours, and preferences of your respondents. Used across many different industries, they can provide you with valuable data when carrying out market or scientific research, providing insight into what your audience feels or thinks.
Generally, a questionnaire is made up of a number of questions with multiple predefined answers for your respondents to choose from. When it comes to creating your questionnaire, you need to think carefully about what you’re wanting to achieve and what information you are looking to gather.
With that in mind, this brief guide is looking to provide helpful tips about how to create, design and optimise your questionnaire for the best results.
Online vs Paper Questionnaires
Choosing between online or paper versions depends on many factors, including your target audience, budget, time constraints and the type of questions you’re asking.
Online Questionnaire
- Generally, lower costs
- Less time consuming
- Better engagement
Online surveys provide more flexibility with the option of using drop-down boxes and allow respondents to type rather than to write their answers. It is also less effort for respondents to simply complete a survey on their phone or desktop and click send, rather than having to post or hand in their paper answers.
However, online surveys can pose accessibility concerns to those who don’t use the web. They also require a level of technical knowledge that not everyone may have, and some researchers believe that while both may be similar in structure, the way participants perceive them differs, with paper questionnaires being more favourable – referring to psychometric considerations. Also some of your participants might not even receive the invitations to complete, due to having multiple and unused emails, or the invitations going to junk folders.
Paper Questionnaire
- Readability to a wider demographic
- A questionnaire can serve as a document
Paper questionnaires do have their strengths, such as providing access to a target demographic that may not have the technical knowledge to complete an online survey. They also provide access to visually impaired, so no matter how positive online questionnaires may seem, there’s no point sending one out if they’re not going to reach your target audience. They also prove extremely useful in meetings or seminars where the respondent can instantly hand their paper questionnaire in.
However, paper questionnaires do increase paper consumption and may not be the most environmentally friendly way of sourcing the data you need. They also have generally larger costs associated with designing and distributing them. The more responses you get, the more data entry you’ll find yourself stuck with – unless you find a company providing document scanning services to take care of that for you.
Developing Questions
- Don’t prompt to a specific answer with your questions
- Use appropriate language to your target audience
- Ask only one question at a time
- Have a logical flow to your questions
When it comes to choosing the questions you’re going to ask, you need to ensure you’re not prompting or leading to a particular answer. You also need to use appropriate language for your target audience and demographic, asking questions in specific ways to ensure you receive rich data that provides you with quality insights. It’s also important to only ask one question at a time.
You first need to consider if you’re going to ask open, closed or scale rating questions to acquire your data. Open questions will allow you to explore topics in more detail and give your respondents the chance to provide a lengthier answer, however, people may not have the time or won’t bother to give a longer answer, possibly invalidating the response. Closed questions, on the other hand, give respondents multiple choice answers, making them easier and faster to complete.
It is also important to ensure that your questionnaire has a logical flow and doesn’t jump back and forth between topics, as it can confuse respondents and provide less valid responses for data capture.
Example of a bad question
This example illustrates several issues that you should always try to avoid when developing questions:
- By using word helpful you create a bias that the representatives were helpful.
- This question is asking two things in one question – whether the needs were accommodated and whether good prices were provided, which is confusing to the respondents as to which question they are answering.
- Although it is not necessarily wrong to use yes and no questions, you might be missing out on potentially richer data.
Example of a good question
- The way this question is asked doesn’t form any biases or prompting for a particular answer.
- Only one question is asked – to rate the service, therefore the respondent will not be confused as in the previous example.
- 1 to 10 rating types of questions provide more flexibility in terms of understanding the data – some choose to ignore all the ratings except 10 and 1, others might consider accounting for broader answers.
Designing/Optimising the Questionnaire for Data Capture
- Carefully plan the survey at the beginning
- Consider visibility and readability (sizes of fonts, checkboxes, etc.)
- Avoid open questions if they are not necessary
First and arguably the most important step is to identify your research aims and develop your questions. It is important to understand that there rarely is a one-size-fits-all type of questionnaire, it really does depend on many factors, however, there are some useful tips you can follow to ensure accurate and efficient data capture after you receive the responses. Once you have optimised each question separately, it’s time to design the overall layout.
Do your best to plan the survey at the beginning thoroughly, so that there won’t need to make any changes during the process, as that can significantly hinder initially set timescales, budget, etc. When designing the overall flow of your questionnaire, take extra care with visibility concerns – use reasonably sized checkboxes and fonts, ensuring that you’re making it as easy as possible for respondents to read and understand the questions. Also, try to avoid using open questions, unless those are crucial for the research, as they make data capture much harder, due to having to capture those manually, rather than using OCR and OMR technologies.
Generally, if you want to ensure that data capture goes as smoothly and cost-effectively as possible and utilises a minimal amount of human intervention aim for consistent and attractive layout. Exceptions, open-ended questions, inconsistencies will not only hinder the data capture process but consequently affect the answers from respondents.
Below are two examples of event feedback surveys, one of them designed with specific goals in mind to receive effective and relevant responses, while other has several issues to demonstrate what you should be looking to avoid.
Example of a badly designed questionnaire
- The overall layout and questions logical flow is clearly not planned well enough, resulting in little to no logical flow of questions.
- Answer options are confusing, for example questions 2, 4 and 6 have very poorly written answers.
- No consistency in font sizes, fonts, checkbox positions, and checkbox sizes. That does not only look very unattractive to respondents, it also complicates the data capture process.
- Question 7 has no real reason to be an open-ended question – it would work much better if it was 1 to 10 rating.
- There are more things that are not well designed, such as starting with a personal question, however, it all comes back to planning and deciding what is important to ask and finding out the best way to ask it.
Example of a well-designed questionnaire
- Questions follow a clear logical flow.
- Questions are thought out to provide relevant data.
- Questions are short and easily understandable.
- Does not include personal questions, however if they are required for your study try to include those at the end.
- This questionnaire mostly utilises rating scales for questions, which are great for small questionnaires, for longer questionnaires it is a good idea to also add closed-ended questions along with rating scales because respondents can just start filling out the same number to finish the survey quicker.
If you would like to start your own research project, but unsure where to start feel free to get in touch with us and we will gladly help you with any survey design, data capture and distribution related questions.
Feature image by janoon028 from freepik.com