Commercial Content
By Pete Cape
Why must we gamify our surveys?
We don’t do it to make them “look pretty” or to improve participant engagement. We gamify to collect better data.
John Krosnick of Stanford University first applied Herbert Simon’s concept of satisficing to survey research. He recognised that survey-taking is not always an easy thing to do. It can be cognitively burdensome. This burden comes from the tasks we ask people to do, particularly tasks of memory and self-reflection.
It is considerably easier to ask, “how many different magazines have you read in the past year and how many copies of each did you read?” than it is to recall the correct answer. What you get is a satisficed answer, an answer that satisfies the task, to some degree, but is not the optimal (full) answer.
How does gamification work to overcome satisficing?
By increasing motivation. To understand this we turn to psychology where we find similar terms to satisficing; apathy and amotivation. In this state of motivation a task will not be done or will not be done correctly – the same outcome as under satisficing.
Deci and Ryan’s Self-Determination Theory tells us precisely how to move people up the motivation continuum towards a state of intrinsic motivation. We need stress four things:
- Autonomy: the participant freely chose to do this
- Relatedness: people like them do it
- Value: it has value
- Competence: that they are good at it
Best way to make someone feel that they are good at something? Praise them for doing well…
Why do researchers find it hard to praise participants?
Researchers recognise that sometimes the challenge posed to participants is not hard. It feels patronising to say “well done” for something that everyone can do. But we must find ways to do it, and gamification offers that way.
How should we gamify our surveys?
Firstly – define the term “game.” A game must have an objective, a set of rules and a mechanic to play – and that’s all. The mechanic normally worries people most, but it shouldn’t. We have a mechanic – the questions we want to ask and the means by which we collect the answers.
All we have to do is reframe the survey question as a game and attach some rules. It’s that simple.
What happens when we gamify our surveys?
We get more data.
We used three different approaches to the question. The first approach was a standard one: we just asked for all the brands of mobile phones the participant knew. We then gamified it by asking for the same information, saying up to a maximum of ten, suggesting then this would be a “winning amount.” We also stated that there would be a 60-second time limit (there wasn’t really) and that “this is a game.” The second game we designed also asked for all the brands that the participant knew, but stated that they needed to get at least five. Note that by placing a limit of the number of brands we potentially ask for less data, not more.
The table below displays the results.
It’s easiest to look at each area of the table separately:
Average number of mentions
This goes up in both the gamified treatments from around four per person to an average of five-and-a-half. The median number also increased from four to five.
People who mentioned “zero brands”
This stays much the same, suggesting perhaps there is some strong satisficing going on that needs a different approach to overcome.
People naming one brand only
In the standard survey, almost one in five elected to give a single brand only – pure satisficing. When gamification is introduced, the numbers saying one brand reduces to a much more reasonable two percent.
People giving exactly five brands
When we change the rule from “a maximum of ten” to “at least five” we run the risk of condoning or encouraging satisficing, by suggesting that five is “good enough.” There is a spike at five in the second gamified version. Nearly a third of people give five brands. However, they come not from people who give six to ten brands, but from those giving one to four brands. Adding his instruction/rule encourages more answers.
People providing more than ten brands
This is where we potentially lose data if we set the rule incorrectly. In the standard version only two percent were able to name more than ten brands. Our design was successfully able to minimise the potential for data loss.
How can we stop participants from feeling demotivated by the game?
Simple, tell them they did well, irrespective of actual performance and relate that to how everyone else is doing. They are motivated just knowing they are competent, and others are doing as well (or as badly).
Is the data gain worth gamifying for?
This is the $64,000 question. We would contend “yes.” The question costs nothing to administer, just some thought in advance and some desk research. The data is better than under the standard condition, and closer to the truth. The “at least five” game produces more data. Perhaps there may be another game that could give us yet more data?
Gamification gives an opportunity to give feedback on performance and praise competency – increasing motivation. The feedback mechanism also tells the participant that someone (or something) is looking at what they are doing, while they are doing it. This serves to reduce satisficing in the non-gamified questions in the same survey.
Pete Cape (AKA “Dr. Pete”) Global Knowledge Director, SSI, is among the most respected, entertaining and thought-provoking presenters in the marketing research industry. Pete has almost 30 years of experience in market research and is a frequent speaker at conferences, seminars and training workshops around the globe as well as a regular contributor to research and marketing publications.
