Over the years, ESOMAR has been a strong supporter of young professionals and their ideas. To encourage industry involvement and sharing, the Young ESOMAR Society (or YES!) was launched and with it, the YES Award.
At Congress 2019 in Edinburgh, Kristen Ball from Kantar took home the winning title and has written about her idea in more detail to share with the world…we hope you enjoy!
Researchers currently rely on a one-size-fits-all approach to combat dishonesty in surveys, but Kantar Profiles knows better! We have been conducting research on research methods for years, which made it relatively easy to identify all the respondent behaviors and responses that researchers still need to minimise.
Meme Honesty Priming is a combination of pictures and words strategically chosen to connect with respondents to positively impact the quality and quantity of survey data. Priming techniques can also include questions, which respondents answer prior to seeing the relevant priming insights.
We consider a survey’s characteristics (i.e. number of questions, type of questions, subject matter, etc) as well as the behavior trends most likely to be observed (i.e. respondent’ habits, reactions, tendencies, etc) to create each survey’s priming narrative.
The narrative is presented prior to entering the survey, to relate with respondents about a common feeling or behavior, to identify the behavior/s we want them to avoid, to identify the best approaches to answering the questions and to achieve our goals of increasing respondent self-awareness and encouraging more realistic responses.
Reasons to not tell the truth
We identified the motivations behind each of the unwanted behaviors and responses and grouped them by category, producing four primary motivations for lying:
- I am lacking a positive attribute: I have a lack of self-awareness, or I want to project a positive image of myself.
- I possess a negative attribute: I have attitudes/opinions others may not agree with, I do not think you are interested in what I really have to say, or I do not know the answer to the question.
- I have a survey mindset: I am in contempt of the survey, I am putting on a persona, I am a survey hacker, I am a bot, or I do not want to be bored.
- My attention is elsewhere: I am bored, or it takes too much thinking for me to work out the answer to the question.
More specific motivations for dishonesty all stem from these primary four. Upon further exploration of those, we noticed other similarities as well, resulting in two primary categories of dishonesty:
Conscious Dishonesty: These people have bad intentions and deliberately harm survey data.
Subconscious Dishonesty: These lies are not malicious in nature because these people are just giving in to subconscious tendencies (they may not even know they are “lying”).
These observations confirm that dishonesty is very complex, meaning the one-size-fits-all approach will never be the most effective solution! Acknowledging the specific lying motivations is necessary for having a real impact on survey data quality, but this is easier said than done. To eliminate as many dishonest responses as possible, researchers need to make respondents aware of their subconscious tendencies without shaming them.
Meme Honesty Priming
We believe our Meme Honesty Priming is the best way to do this. Memes are customizable, and people already know how to engage with them. They are also perfect for communicating sensitive messages because they are funny, relatable and the use of imagery allows us to say so much more while saying less (which is essential for keeping the length of interview low).
We conducted multiple rounds of testing to confirm the usefulness of our Meme Honesty Priming. We first tested this method on a Kantar Worldpanel study, in which respondents were asked questions about their grocery shopping habits, environmental and political opinions, embarrassing questions, and we added a few open-ends as well.
We assessed the survey’s content to identify the specific tendencies respondents would most likely have while completing that particular survey. From there, we were able to create the customized Meme Honesty Priming narrative, which equipped respondents with the awareness necessary to resist the identified tendencies (Figures 1 and Figure 2 are examples of memes used in this study’s Meme Honesty Priming narrative).
As you can see in Figure 3 (below), when respondents were not primed, they tended to overclaim virtuous activities and behaviors and underclaim less virtuous activities and behaviors compared to the data collected from the primed respondents.
Source: Kantar
Additional factors
To maximize the effectiveness of this method, we then adapted our research to test three additional variables:
- Cultural relevance of the images and narrative
- The location of the priming
- The subject of the images
The second round of testing was done on a GroupM LIVE Panel study across four markets, in which we asked respondents about their purchase activities and media consumption. We made several versions of the survey to test the effects of each variable individually. These were our findings:
1. Cultural Relevance: There is a positive correlation between the effectiveness of Meme Honesty Priming and the extent to which a respondent relates to the narrative used (see Figure 4).
2. Location: To maximize effectiveness, Meme Honesty Priming should be placed up front and supported by mid-survey boosters as needed (see Figure 5).
3. Subject: Researchers should manipulate this variable to intensify the thought or emotion portrayed by the narrative (see Figure 6).
Simply put, the most effective narratives are the most relatable ones.
Confirming our findings
We conducted one more round of testing on a Disney diary study to verify our findings and the overall consistency of Meme Honesty Priming’s effectiveness. In this study, respondents (parents) were asked to report on their children’s daily activities. The narrative was created with culturally relevant jargon and images and the content was as specific to the survey as possible. Mid-survey boosters would have been incorporated as well, but this was a relatively short survey, so they were not needed.
The results of this study mirrored our findings from the first two rounds of testing. After being exposed to our priming narrative, we observed a drop in the reported participation in virtuous activities and an increase in the reported participation in less virtuous activities (see Figure 7).
Meme Honesty Priming consistently impacts survey data by giving respondents the awareness they need to answer questions more honestly than they would have otherwise. This method has proven its effectiveness across cultures and languages and it can be customized to support any survey topic. Through the implementation of Kantar Profiles’ Meme Honesty Priming, clients can now minimize the collection of inaccurate data by targeting subconscious dishonesty before it becomes problematic.
The source of all charts and figures is Kantar.
Does YES peak your interest? Want to be a part of the community? Or know someone who would? Be sure to check out the YES website and get yourself and/or colleagues 30 and under involved! Speaking, competing for an award, programme committees, becoming published, volunteering as a student…it’s all possible with YES!
