14 Conseils pour créer un bon questionnaire
Before you can post your survey on SurveyCircle to find participants, you first need to create your survey with a survey software of your choice (e.g. Qualtrics or SurveyMonkey). After creating your questionnaire, you will receive a link to your survey which you will need later to post your survey on SurveyCircle.
To ensure that you eventually get all the data you need for answering your research question, we have put together 14 tips for creating a good survey:
When creating your questionnaire, you should put yourself in the shoes of your participants, as they are the ones who must understand and answer your questionnaire.
Therefore, it can be helpful to look at the cognitive process that people normally go through when answering questions:
Your goal should be to keep this cognitive process as simple as possible. If the cognitive effort for your participants is too high, this may lead to biased responses and systematic deviations in your participants’ response behavior (response tendencies). Eventually, this can make your data useless.
Response tendencies occur whenever the answers given by a participant in a questionnaire systematically deviate from the participants’ actual opinions. This is often a consequence of satisficing, which means that a participant chooses the first best answer and not the most accurate one. On the one hand, satisficing reduces the cognitive effort, but on the other hand, it reduces the quality of your data as well.
Examples of other response tendencies are the acquiescence tendency (tendency to agree) and the tendency towards the middle.
Both content and structure of your survey are essential for its quality.
On the first page of your questionnaire you should welcome your participants with a short introduction. It should contain some basic information, e.g. the topic and goal of your survey as well as information about yourself, your university, the duration of your survey, and possible incentives. You should also include information on how you intend to use the data. By addressing your participants in a friendly manner, you can build a positive relationship with them and increase their willingness to complete your survey in a focused way.
The main part of your survey, i.e. your actual questionnaire, should be structured by topic: Start by asking all questions on topic no. 1 and then continue with all questions on topic no. 2, etc. By structuring your questionnaire that way, you make it easier for your participants, because they don’t have to jump back and forth between different topics.
Demographic data should ideally be collected at the end of the survey. These questions usually require only little cognitive effort and are rarely skipped – and therefore they are good questions to end your survey with. On the very last page of your questionnaire, make sure to thank your participants for their participation.
Try to keep the duration of your survey as short as possible. If your survey contains questions that are only relevant to a subset of your participants, you can use filters to make sure these questions are only displayed to the right audience.
You should also avoid questions that do not contribute to answering your research question. These questions would unnecessarily lengthen the duration of your survey. Before putting a question in your questionnaire, always ask yourself how you would use the data obtained and whether you really need it.
If your questionnaire contains relevant questions only, your respondents will usually remain more focused and you will receive more thoughtful answers. You should also be rewarded with lower drop-out and non-response rates.
A progress bar within your questionnaire can also motivate your participants. You can either indicate the progress in percent or the number of remaining questions. Knowing how far they have come or how many questions they still have to answer will help them stay focused and reduce the likelihood of them dropping out of your study.
Design your questionnaire as user-friendly as possible and avoid unnecessary technical features. Your survey should be compatible with all common devices (laptop/desktop computer, tablet, smartphone) and browsers so that as many people as possible can participate. It is usually best to choose a responsive and user-friendly design that allows your participants to focus on the actual questions.
Before you publish your survey, you should conduct a pretest in order to identify comprehension problems and typical mistakes before the actual survey period begins. You can ask your family, friends or fellow students to fill out your questionnaire and give you feedback afterwards. Face-to-face pretests, where your pretesters give you live feedback while filling out the questionnaire, can also be very useful.
If you use filter questions in your survey, make sure to test the filter logic several times to make sure your participants see the right questions and are assigned to the correct path within the questionnaire.
After the pretest, you can revise your questionnaire and then publish it through your survey tool. This will generate a link to your survey that you can use to post it to SurveyCircle.
Is your study already online? Find out how to get as many participants as possible.
Make sure to phrase your questions in a simple, clear and comprehensible way. Avoid abstract and ambiguous terms which make it difficult to answer your survey. You should also avoid technical and unfamiliar terms, foreign words, and abbreviations. If a technical term is unavoidable, try to explain it by using a definition so that your participants know exactly what you are referring to. If your questions are not properly understood by your audience, this can lead to arbitrary answers or to participants dropping out of your survey.
Example: "To what extent do you agree with the following statement?"
Bad: Double-barreled question
Better: Split the statement
Answering a question is difficult when you include different stimuli into one question but your participants want to answer to each of the stimuli in a different way.
This may lead to your participants not knowing how to answer the question properly and skipping it instead.
In the example above, it is possible that some people may find the participant recruitment with SurveyCircle quite easy, but it took them some time to find many participants. These people would not be able to answer the double-barreled question properly. Likewise, it would not be possible for you as a Survey Manager to evaluate the two stimuli individually.
To avoid this problem, split the statement or question into individual stimuli and record the answer of your participants for each stimulus individually.
Example:
Bad: Leading question
Better: Neutral phrasing
Leading or suggestive questions are questions that suggest a certain answer to the participants. This can lead to biased responses as you prompt the participants to answer the question in a particular way.
In order to obtain valid answers, you should phrase your questions as neutrally as possible without giving any implicit recommendations.
Example:
Bad: Imprecise reference period
Better: Precise reference period
If you refer to a specific time frame in your question, try to define it as precisely as possible. You should avoid terms such as recent, lately or earlier, as they can be interpreted in many different ways. To be able to compare the answers of your participants in the best way possible, the time frame mentioned in your question must be understood equally by all participants.
You should also make sure that the time frame in your question is appropriate for the question’s content. If you ask questions about frequent or daily events (e.g. doctor’s appointments or meals), the time frame in your question should be rather narrow so your participants can easily answer the question without having to estimate frequencies. If you ask about rare or non-daily events (such as moving or a wedding), you may also use a broader time frame in your question as respondents usually remember such events well.
Example:
Bad: Double negation
Better: Positive phrasing
Statements with multiple negations can be confusing for your participants and may lead to comprehension problems. Responding to a negatively worded statement on a Likert Scale requires high levels of concentration and great cognitive effort. Therefore, try to avoid double negations whenever possible.
Example: To what extent do you agree with the following statement: "SurveyCircle helped me recruit survey participants."
Bad: Unbalanced response scale
⚪ strongly agree
⚪ agree
⚪ rather agree
⚪ disagree
⚪ strongly disagree
Better: Balanced response scale
⚪ strongly agree
⚪ agree
⚪ rather agree
⚪ neither agree nor disagree
⚪ rather disagree
⚪ disagree
⚪ strongly disagree
When creating your survey, make sure to use balanced response scales. This means that your response scale has an equal number of positive and negative answer options. If there are more positive than negative answer options, respondents tend to choose one of the positive answer options because their superior number implies an implicit recommendation (see the section on leading questions).
Ideally, the response scales used in your survey should include a neutral answer category, too, such as I don't know, I neither agree nor disagree or not applicable, so that participants who have no opinion on a particular topic or are indifferent about their opinion can express it properly.
Example: "How many SurveyCircle points have you collected so far?"
Bad: Answer options overlap
⚪ 0 points
⚪ 0-100 points
⚪ 100-200 points
⚪ 200-300 points
⚪ more than 300 points
Better: response options are disjoint
⚪ 0 points
⚪ 1-100 points
⚪ 101-200 points
⚪ 201-300 points
⚪ more than 300 points
Your answer options should be disjoint, i.e. they should not overlap with each other. In our bad example above, people with exactly 0, 100, or 200 SurveyCircle points would each fit into two of the answer categories given. This makes answering the question impossible for these participants – and it makes it quite difficult for you to interpret the data during data analysis.
Example: "When did you post your first survey on SurveyCircle?"
Bad: Incomplete list of answer options
⚪ 2016 or earlier
⚪ 2017
⚪ 2018
⚪ 2019 or later
Better: Complete list of answer options
⚪ 2016 or earlier
⚪ 2017
⚪ 2018
⚪ 2019 or later
⚪ I have never posted a survey on SurveyCircle
To get valid answers, your answer options need to cover all the answers your participants might want to give. Not finding the appropriate answer option can be very frustrating for your participants.
In order to avoid missing answer options in your survey, it can be helpful to have a heterogeneous group of subjects answer the questionnaire before you finalize it. They can point out missing answer categories that you had not considered yet.
To keep your survey structured, order your answer options logically. When using Likert Scales you should arrange the answer options from the lowest degree of approval (left) to the highest degree of approval (right). If you are asking for numerical values, arrange them in increasing or decreasing order.
A chaotic or inconsistent arrangement of answer options may cause your participants to accidentally choose the wrong answer.
By varying question and answer types in your questionnaire from time to time, you can avoid boring your participants. Also, they won’t get too familiar with your question types. This is good for your data quality because a high degree of familiarity with question and answer types can lead to some participants rushing through your questionnaire instead of completing it carefully.
→ Join SurveyCircle as a Survey Manager
→ How to present your study on SurveyCircle in the best possible way
→ How to find lots of participants for your survey
To ensure that you eventually get all the data you need for answering your research question, we have put together 14 tips for creating a good survey:
General tips
Put yourself in the shoes of your participants
When creating your questionnaire, you should put yourself in the shoes of your participants, as they are the ones who must understand and answer your questionnaire.
Therefore, it can be helpful to look at the cognitive process that people normally go through when answering questions:
- Understanding the question
- Retrieving relevant information from memory
- Using retrieved information to form a judgment
- Response)
Your goal should be to keep this cognitive process as simple as possible. If the cognitive effort for your participants is too high, this may lead to biased responses and systematic deviations in your participants’ response behavior (response tendencies). Eventually, this can make your data useless.
Response tendencies occur whenever the answers given by a participant in a questionnaire systematically deviate from the participants’ actual opinions. This is often a consequence of satisficing, which means that a participant chooses the first best answer and not the most accurate one. On the one hand, satisficing reduces the cognitive effort, but on the other hand, it reduces the quality of your data as well.
Examples of other response tendencies are the acquiescence tendency (tendency to agree) and the tendency towards the middle.
Structure your survey
Both content and structure of your survey are essential for its quality.
On the first page of your questionnaire you should welcome your participants with a short introduction. It should contain some basic information, e.g. the topic and goal of your survey as well as information about yourself, your university, the duration of your survey, and possible incentives. You should also include information on how you intend to use the data. By addressing your participants in a friendly manner, you can build a positive relationship with them and increase their willingness to complete your survey in a focused way.
The main part of your survey, i.e. your actual questionnaire, should be structured by topic: Start by asking all questions on topic no. 1 and then continue with all questions on topic no. 2, etc. By structuring your questionnaire that way, you make it easier for your participants, because they don’t have to jump back and forth between different topics.
Demographic data should ideally be collected at the end of the survey. These questions usually require only little cognitive effort and are rarely skipped – and therefore they are good questions to end your survey with. On the very last page of your questionnaire, make sure to thank your participants for their participation.
Keep your survey short
Try to keep the duration of your survey as short as possible. If your survey contains questions that are only relevant to a subset of your participants, you can use filters to make sure these questions are only displayed to the right audience.
You should also avoid questions that do not contribute to answering your research question. These questions would unnecessarily lengthen the duration of your survey. Before putting a question in your questionnaire, always ask yourself how you would use the data obtained and whether you really need it.
If your questionnaire contains relevant questions only, your respondents will usually remain more focused and you will receive more thoughtful answers. You should also be rewarded with lower drop-out and non-response rates.
A progress bar within your questionnaire can also motivate your participants. You can either indicate the progress in percent or the number of remaining questions. Knowing how far they have come or how many questions they still have to answer will help them stay focused and reduce the likelihood of them dropping out of your study.
Focus on clarity and compatibility
Design your questionnaire as user-friendly as possible and avoid unnecessary technical features. Your survey should be compatible with all common devices (laptop/desktop computer, tablet, smartphone) and browsers so that as many people as possible can participate. It is usually best to choose a responsive and user-friendly design that allows your participants to focus on the actual questions.
Pretest your survey
Before you publish your survey, you should conduct a pretest in order to identify comprehension problems and typical mistakes before the actual survey period begins. You can ask your family, friends or fellow students to fill out your questionnaire and give you feedback afterwards. Face-to-face pretests, where your pretesters give you live feedback while filling out the questionnaire, can also be very useful.
If you use filter questions in your survey, make sure to test the filter logic several times to make sure your participants see the right questions and are assigned to the correct path within the questionnaire.
After the pretest, you can revise your questionnaire and then publish it through your survey tool. This will generate a link to your survey that you can use to post it to SurveyCircle.
Is your study already online? Find out how to get as many participants as possible.
Tips for phrasing questions and answer options
Make sure to phrase your questions in a simple, clear and comprehensible way. Avoid abstract and ambiguous terms which make it difficult to answer your survey. You should also avoid technical and unfamiliar terms, foreign words, and abbreviations. If a technical term is unavoidable, try to explain it by using a definition so that your participants know exactly what you are referring to. If your questions are not properly understood by your audience, this can lead to arbitrary answers or to participants dropping out of your survey.
Examples for phrasing questions and answer options
Avoid double-barreled questions
Example: "To what extent do you agree with the following statement?"
Bad: Double-barreled question
- "Finding participants with SurveyCircle is quick and easy."
Better: Split the statement
- "Finding participants with SurveyCircle is easy."
- "Finding participants with SurveyCircle is quick."
Answering a question is difficult when you include different stimuli into one question but your participants want to answer to each of the stimuli in a different way.
This may lead to your participants not knowing how to answer the question properly and skipping it instead.
In the example above, it is possible that some people may find the participant recruitment with SurveyCircle quite easy, but it took them some time to find many participants. These people would not be able to answer the double-barreled question properly. Likewise, it would not be possible for you as a Survey Manager to evaluate the two stimuli individually.
To avoid this problem, split the statement or question into individual stimuli and record the answer of your participants for each stimulus individually.
Don't use leading questions
Example:
Bad: Leading question
- "Don't you think that SurveyCircle is a good way to recruit study participants?"
Better: Neutral phrasing
- "How do you rate SurveyCircle as a way to recruit study participants?"
Leading or suggestive questions are questions that suggest a certain answer to the participants. This can lead to biased responses as you prompt the participants to answer the question in a particular way.
In order to obtain valid answers, you should phrase your questions as neutrally as possible without giving any implicit recommendations.
Use precise and reasonable reference periods
Example:
Bad: Imprecise reference period
- "Have you recently participated in a study on SurveyCircle?"
Better: Precise reference period
- "Have you participated in a study on SurveyCircle in the last three days?"
If you refer to a specific time frame in your question, try to define it as precisely as possible. You should avoid terms such as recent, lately or earlier, as they can be interpreted in many different ways. To be able to compare the answers of your participants in the best way possible, the time frame mentioned in your question must be understood equally by all participants.
You should also make sure that the time frame in your question is appropriate for the question’s content. If you ask questions about frequent or daily events (e.g. doctor’s appointments or meals), the time frame in your question should be rather narrow so your participants can easily answer the question without having to estimate frequencies. If you ask about rare or non-daily events (such as moving or a wedding), you may also use a broader time frame in your question as respondents usually remember such events well.
Don't use double negations
Example:
Bad: Double negation
- "It wouldn't be good if you weren't in a quiet place while answering the survey."
Better: Positive phrasing
- "It is important that you are in a quiet place when answering the survey."
Statements with multiple negations can be confusing for your participants and may lead to comprehension problems. Responding to a negatively worded statement on a Likert Scale requires high levels of concentration and great cognitive effort. Therefore, try to avoid double negations whenever possible.
Make sure to use balanced response scales
Example: To what extent do you agree with the following statement: "SurveyCircle helped me recruit survey participants."
Bad: Unbalanced response scale
⚪ strongly agree
⚪ agree
⚪ rather agree
⚪ disagree
⚪ strongly disagree
Better: Balanced response scale
⚪ strongly agree
⚪ agree
⚪ rather agree
⚪ neither agree nor disagree
⚪ rather disagree
⚪ disagree
⚪ strongly disagree
When creating your survey, make sure to use balanced response scales. This means that your response scale has an equal number of positive and negative answer options. If there are more positive than negative answer options, respondents tend to choose one of the positive answer options because their superior number implies an implicit recommendation (see the section on leading questions).
Ideally, the response scales used in your survey should include a neutral answer category, too, such as I don't know, I neither agree nor disagree or not applicable, so that participants who have no opinion on a particular topic or are indifferent about their opinion can express it properly.
Avoid answer options that overlap
Example: "How many SurveyCircle points have you collected so far?"
Bad: Answer options overlap
⚪ 0 points
⚪ 0-100 points
⚪ 100-200 points
⚪ 200-300 points
⚪ more than 300 points
Better: response options are disjoint
⚪ 0 points
⚪ 1-100 points
⚪ 101-200 points
⚪ 201-300 points
⚪ more than 300 points
Your answer options should be disjoint, i.e. they should not overlap with each other. In our bad example above, people with exactly 0, 100, or 200 SurveyCircle points would each fit into two of the answer categories given. This makes answering the question impossible for these participants – and it makes it quite difficult for you to interpret the data during data analysis.
Use complete lists of answer options
Example: "When did you post your first survey on SurveyCircle?"
Bad: Incomplete list of answer options
⚪ 2016 or earlier
⚪ 2017
⚪ 2018
⚪ 2019 or later
Better: Complete list of answer options
⚪ 2016 or earlier
⚪ 2017
⚪ 2018
⚪ 2019 or later
⚪ I have never posted a survey on SurveyCircle
To get valid answers, your answer options need to cover all the answers your participants might want to give. Not finding the appropriate answer option can be very frustrating for your participants.
In order to avoid missing answer options in your survey, it can be helpful to have a heterogeneous group of subjects answer the questionnaire before you finalize it. They can point out missing answer categories that you had not considered yet.
Arrange answer options in a logical way
To keep your survey structured, order your answer options logically. When using Likert Scales you should arrange the answer options from the lowest degree of approval (left) to the highest degree of approval (right). If you are asking for numerical values, arrange them in increasing or decreasing order.
A chaotic or inconsistent arrangement of answer options may cause your participants to accidentally choose the wrong answer.
Vary question and answer types
By varying question and answer types in your questionnaire from time to time, you can avoid boring your participants. Also, they won’t get too familiar with your question types. This is good for your data quality because a high degree of familiarity with question and answer types can lead to some participants rushing through your questionnaire instead of completing it carefully.
→ Join SurveyCircle as a Survey Manager
→ How to present your study on SurveyCircle in the best possible way
→ How to find lots of participants for your survey