Survey respondents, whether consciously or unconsciously, can claim to have done more, seen more, or heard of more than they really have. It’s a behaviour called overclaiming. Often it comes down to the nature of memory; it’s easy to think you’ve seen something when you’re prompted. The conversation tends to play out like this: Seen this brand of chocolate bar before? “Hmm… maybe I have, it rings a bell, best to say yes just in case.” In a minority of cases survey respondents can deliberately say they’ve seen more than they have, to gain access to the survey. Both of these types of answers aren’t useful for you, as a survey creator; they’re dishonest, whether intentionally or not.

There’s plenty you can do, though, as a survey creator to reduce overclaiming behaviour from your respondents with some simple tweaks to your survey design. It’s a give and take relationship with your survey sample; by making some simple adjustments to your next survey, you can encourage the best quality, most truthful answers from respondents. Rather than littering your survey with “trap” questions, treat your respondents with respect and they will reward you with honest answers in return.

In this article we talk through five simple ways to write a survey that encourages honest, thoughtful answers. Attest Customer Research Team member, Nikos, recently implemented these practices in a client’s survey and saw overclaiming behaviours drop by almost 30%.

Avoid leading titles

Survey respondents want to take your survey, to get the rewards on offer and to share their opinion with you. By naming the survey after your brand, your products or even referencing the topic of the survey, respondents are naturally going to gear their responses to fit. They don’t want to be qualified out when they feel they have value to share with you. Whether consciously or not, they may tailor their responses to what they believe you want to hear, and the name of your survey can give a big clue as to what you want to hear! 

Make sure your survey is named something interesting but, crucially, nothing that leads or biases respondents. Avoid including your brand name, naming your sector or the consumers you want to hear from. For instance: “Laithwaites”, “Wine & alcoholic drinks survey” and “Wine-drinkers research” are all biasing names for your survey. In that case “Your drinking habits” would be a much less biasing title – and avoids the added negative of deterring non-wine-drinkers who you may be interested in hearing from too! Use the internal title of your survey, which isn’t visible to respondents, to name the survey in a recognisable way for your team, instead.

Talk to your respondents, don’t grill them

Opening your survey with a text card can set good expectations, give necessary context and take the pressure off the respondent to answer in an expected way. 

Fundamentally, your survey should be a two way conversation between your brand and your consumers. You want answers to your questions, but no one likes to feel grilled. Speaking to your respondents as a human, and encouraging them to see you in the same light can have a marked impact on engagement levels and promote honest answers. 

Text cards also help break up longer surveys and give respondents a brief respite from questions, which can help them stay engaged for questions asked later in the survey. Make sure to use them to signpost how far through the survey your respondent is, and thank them for their honest answers. 

Remove agreement bias

A really easy way to bias your respondents into giving a certain answer is to ask a question with “Yes” and “No” answers. We’re hardwired to agree when presented with these types of questions, it’s human nature. Agreement bias (also known as acquiescence bias) has such a noticeable impact on survey answers that the Attest platform will flag to you when you’ve asked a question that is likely to suffer from agreement bias. 

It’s easy to rephrase any single or multiple choice question to avoid agreement bias. Simply make the subject of the question into an answer option, and ask a broader question;

Which of these items do you own?

A car / Your home / A motorbike / A boat 

In combination with the other tips listed, this shift can encourage greater consideration and honesty from your survey respondents. For instance, when you change this question, make sure your survey title doesn’t give away that you’re interested in car ownership!

Give respondents a way to say no

If you remove your respondents’ option to tell you that the question doesn’t apply to them, they don’t understand what is being asked or they would rather not answer, you’re forcing them to give you a poor quality answer. Every question has additional answer options you can choose to toggle on or off, including “None”, “N/A” and “Other”. Use these options to give your respondent a chance to answer honestly. 

Every respondent also has the option to skip the question, in case you decide to toggle the additional answer options off. In many cases, bad data is more dangerous than no data, so you cannot turn off the skip functionality on your survey. We will, however, remove respondents that skip too many questions. 

Think twice before trapping respondents

It’s not always the respondent who is at fault when overclaiming occurs; they want to impress you and complete your survey, so think about what you can do to get the best out of them. Trapping respondents with fake brands or incorrect answer options shows little respect for them, and where the incorrect answers are obvious, they can erode trust in the relationship you have with them, removing the incentive for honesty throughout the rest of the survey. 

And in many cases, memory isn’t infallible. If the respondents have been encouraged to answer honestly using the methods above, they’re more likely to stop and reconsider whether they have or haven’t heard of the fake brand you’re presenting them with. But without the methods used above, respondents are likely to fall into your ‘traps’. 

Next steps

Our Attest centre of excellence team can help you write elegant surveys that encourage honest, thoughtful responses from your respondents…and avoid the pitfalls we outlined above. You’ll get more accurate data as a result, and draw out more meaningful insights. 

Don’t hesitate to get in touch for personal help from the Customer Research Team. You can also check out our Complete guide to survey creation for even more advice from the Customer Research Team on writing surveys to avoid bias, keep engagement high, and promote data quality.