By Alex Wheatley
In any context there are many questions with factual answers which are difficult to answer; “have you considered an affair?”, “how many vegetables do you eat?”, “how often do you go to the gym?”, “have you lied to your boss?” to name just a few.
When it comes to a question like “how much do you drink” it can be hard enough to be honest to ourselves let alone a researcher! Fortunately the anonymity and context of online research puts it in a unique position to secure honest answers to sensitive questions; however this is no easy feat. When we ask a question there are many hurdles we must overcome to reach an honest answer.
Some of these are unique to online research. If we ask someone ‘which of these do you own?’ for instance, there’s an array of biasing factors that can impact our responses:
- acquiescence bias creates over-claim simply because we like to say yes
- you may have an owned an item in the past or plan to buy one, that’s basically a yes… right?
- you could be curious to see what questions come up if you click something you like
- you’ve got something close, I’m sure that counts…
- there’s the dreaded miss-click
- or sod it I’ll lie, why not! I’m sure my answers will still be interesting for them
Moreover, in any context there are a plethora of barriers to the truth that aren’t unique to survey research and go beyond simple questions of ownership. Some questions are embarrassing to answer, some are tricky to put a number on and others can be hard to remember.
In a survey these issues can be further exacerbated by privacy concerns and the desire to get through the survey quickly. Whatever the issue the result however is the same – we cannot take an honest answer for granted.
To help the researcher in navigating this minefield Lightspeed GMI embarked on a multination project to establish the best means of securing honest answers. By experimenting with different methodologies and observing in the data the amount of confessions we elicited, we were able to evaluate which produced the most honest responses. Providing a toolset to exploit the online survey’s unique ability to illicit an invaluable level of honesty.
Our findings divided the issue into three main categories:
- ‘Questions we’re scared to tell the truth about’
- ‘Questions we lie to ourselves about’
- ‘Questions we want to lie about’, each category presenting its own set of issues
The investigation into these three categories utilised: a series of sensitive question such as “Do you always wash your hands in the toilet?”, the self-observation question of ‘How much alcohol do you drink?’ and the over-claim tempting question “Do you own a tablet?”.
The conclusion of our theory and research has been the development of the following three doctrines:
- Questions we’re scared to tell the truth about
When you ask someone a sensitive question that they might be scared or embarrassed to answer, the determining factor in whether they feel comfortable giving you a truthful answer is your relationship with them. In a survey you only have one opportunity to build a rapport that puts your respondent at ease, and that’s the introduction. Our research found that the key to getting people to answer sensitive questions is to engage them emotively and firmly with a powerful introduction. The most successful approach for us was one which broke down our key messages of trust framed within the challenge; “Can you answer our questions as if no one was listening?” and followed by the pledge “I promise to tell the truth!”. We’re not suggesting every survey should have a pledge; but ultimately our approach was much more effective than a traditional introduction because it didn’t make our respondents ever think ‘what is that to you?’ when asked a sensitive question!
- Questions we lie to ourselves about
The self-deception at play when you ask respondents to make an observation about their behaviour is not something that a strong rapport can solve. In fact, our research found the barriers to truth here are not necessarily a matter of honesty or deception at all. The biggest obstacle a respondent faces when trying to make an accurate self-observation is simply calculating the value. We get the most truthful data not when we attempt to combat dishonesty; but when we break down the task to make it simple for the respondent to evaluate, and by giving a time frame rather than burdening the respondent with ambiguous task of averaging, a task that should be the researcher’s anyway!
- Questions we want to lie about
When asking questions which might tempt respondents to actively lie we go beyond the truth invoking powers of a strong rapport. Instead we need to be craftier and ask the question without them realising we have asked it! We got the most truthful responses to our tablet ownership question when we disguised the question by asking it less directly. Asking instead if they wanted a tablet first; then if they said ‘yes’ asking if this was to replace an old one, and if they said ‘no’ asking if this was because they already had one. However, our research showed that while we should ask our question in a less direct form, we must remember to still ask it directly; avoid the temptation to utilise a question format which does not force an answer as component based selection biases strongly influence data.
Alex Wheatley, Research Innovator, Lightspeed GMI
4 comments
thank u i want copy that on my weblog
Yes,definitely these are some of the guidelines which we need to follow while conducting a research.
I had observed:
1.People are more honest when they are spending on an average 20 minutes on a research.If its less people will complete just to fill or if it is more than that either that survey is left or filled with anything which comes in their mind.
2.Most Reliable method i believe is CATI ( Computer assisted Telephonic Interview ) in research as you when you are connected to a person you are asking on spot and registering responses instantly so reduces the chances of fake responses.
3.The amount paid for completing should be good or else it will not yield any thing.
4.Target audience should be selected carefully or else a street vendor will be filling his responses for a feed back on services of 5 star hotels.
I have a hard time saying that people aren’t honest. I believe people are basically honest. They don’t generally intend to lie or want to lie. Human nature means that we don’t always understand what or why we’re doing something, and thus our answers don’t always match reality perfectly. Our role as researchers is to understand how our brains trick us and then create measurement tools that can accommodate those tricks. It’s not just people answering surveys that deal with this tricks, it’s every last one of us, researchers included.
honesty is the important factor for effective survey, thank you