Alastair Gordon
What if we could conduct surveys without asking any questions? What if we could directly assess ‘feelings’ without fancy scales or complex biometric or neuroscience equipment? Strange as these ideas may seem, new technologies offer up exactly that prospect, and – utilised cleverly – they might just be what is needed to ‘save’ conventional market research from the doom increasingly being predicted for our industry. In this article we take a look at some exciting results from facial imaging work being done by Gordon & McCallum and a Swiss technology company, nViso, and suggest ways that methods of this sort could reinvent and reinvigorate traditional research.
Despite many challenges questionnaires remain the bedrock of most market research. While social media research, MROC’s and neuroscience tests all offer useful alternatives, sometimes asking a sample of people is simply the easiest, most cost-effective and most reliable way to tap into consumers’ mind-sets. Yet response rates are declining and, as marketing focuses more and more on the emotional, the problem of getting ‘accurate’ response to survey questions becomes more acute. Surveys are often far too long, too boring, and too lacking in the subtlety necessary to uncover hidden, non-rational response. Most researchers are very aware of these issues and, as a result, debate on methods of emotional measurement and ways of making surveys more engaging rage throughout the industry.
I’d like to propose two thoughts. Firstly, there are simply quite a few limits on how engaging a survey can actually be. Making it really “fun” takes up a lot of creative effort and is seldom fully successful. Design effort might often be better spent on keeping a survey short and simple, rather than on inventing ‘engaging’ approaches. Secondly, there is no foolproof way of getting people to read words, make choices between images or move objects around a screen that does not involve heavy use of their frontal/pre-frontal cortex. While some survey procedures can be shown to be better than others at evoking subconscious reaction, none are really direct measures of emotional response. They are, in one way or another, all implicit measures that are subject to conscious filtering and can not capture direct emotional response.
But what if you could accurately record emotional response without any questions at all? Then you could save the bulk of the survey for rational questions (usage patterns, location, demographics etc.). The survey would be both shorter, and provide more accurate measures of emotional response! Over the last year Gordon & McCallum and nViso have been engaged (together with several helpful clients) in testing this idea using a method called Facial Imaging.
Facial Imaging is based on the work of Dr Paul Ekman (of “Lie To Me” fame) and others, which shows that micro-expressions on our face accurately reflect emotions like Happiness, Surprise, Disgust and Fear. Moreover, these fleeting expressions are both innate (we’re born with them) and expressed in the same way across cultures and ethnicities. Measuring response to facial expressions is not new in Market Research; companies in the USA and Europe have been doing it for some time, either using indirect measures (getting people to associate stimuli with faces with known emotional connotations), or by having expert coders manually encode videos of people responding to marketing material. What nViso has added to these methods is a highly intelligent software system that automates the recognition of the most transitory of micro-expressions, and the ability to collect the data using very basic webcams. Essentially this means that if people can be persuaded to look at a computer screen and turn on their webcam, their emotional response to whatever they are watching can be seen. The figure below (Fig 1), illustrates the setup:
To illustrate what this might mean for research, here’s a couple of examples of output slides from a study of a television ad undertaken in Germany and Switzerland with the support of Switzerland’s LINK Institute.
Now the first slide (Fig. 2) looks at first glance like ‘just another’ TV advertising score – resembling a measure of Persuasion or Engagement etc. But there are major differences. The first is that provides a scientifically measured, passively recorded measure of the amount of emotion actually evoked. Essentially this says that there is a decisive and measureable lift in the amount of underlying emotional response generated by the La Mobilière ad compared to people’s state prior to viewing the ad. The second point is that it is measured directly, not via extrapolation from questioning. This means that this kind of measure can be applied easily across cultures, languages and types of visual materials. The same techniques and measures could be applied to gauge reaction to a pack or an ad-test, across sectors as diverse as banks or toilet tissue, or media from movie trailers to store displays. This strength, combined with the fact that no scales are involved, means that this form of measurement is transportable in a way that research methods have seldom been before.
The second slide (Fig 3). digs down and examines how the emotional response to the commercial evolved on a second by second basis. La Mobilière is a European Insurance company with a record of creating humorous, well executed commercials and, as can be seen here, happiness built well and evoked the intended response as the story-line of the commercial unfolded. Alas, not all the commercials we’ve looked are so successful, but this sort of analysis lets clients easily see where the low and high points occur and what “mix of emotions” are evoked at any point in the ad.
From a research perspective we are once again dealing with a direct measure of the amount of emotional change, but there are two other key points about this output. Firstly note the extreme granularity of response – we can link precise emotional response to very specific images in the commercial. Since there is no respondent task involved, there is no particular limit to the length of the stimulus that can be analysed: we are currently testing it with logos shown for only a few seconds, and while there may be operational constraints in collecting data for an entire movie (we have not tried it yet!) it seems likely this will be possible at some stage soon.
The other point here is that we are automatically collecting six data-points (emotions) each second – – and this is linear data that is objectively and consistently measured by a “machine” with no human intervention. Not only is the quantity and specificity of information improved, but a lot of respondent error and variation is removed. The advantages of this method compared to trying to extrapolate “happiness”, or “surprise” from conventional Likert or similar ordinal rating scales is obvious. The system produces a mass of specific data, very simply.
This can obviously create complications if you have to deal with the raw data for unusually large projects (I’m not looking forward to anyone asking us to assess the next ‘Lord of The Rings’). More importantly though, it also offers up huge opportunities for advanced modelling and analytics, understanding how differing kinds of imagery interact and for sophisticated predictive experiments based only on people watching a few videos. Emotional response data of this sort is thus on a different level of sophistication than most of the “feeling” and rating data that we researchers are used to dealing with.
Essentially all this means that facial imaging holds up the promise of solving some fundamental issues with collection of emotional response, for instance that it can:
a) Overcome the issues of “rationalised” or superficial response to survey questions
b) Deliver consistent response across cultures and ethnicities
c) Work consistently across all sorts of sectors, subjects and media types with little modification.
These three big claims carry in turn the implication that reliance on norms and experience databases can be lessened, since post-exposure measures can easily be contrasted with each respondent’s state prior to seeing the material and issues of cultural impact on scales or question wordings are removed from the equation.
Clearly, these are big claims, and you may or may not yet accept them or the theory behind facial imaging (although we’d urge anyone interested to find out more about this fascinating subject). We are not going to explain or interpret all the numbers in the charts above – it’s not our purpose to evangelise for nViso’s specific set of measures. Rather I just want to point out that systems like facial imaging are reaching the stage where they are very well developed and are gaining significant client interest. As they gain ground, and I believe they will, they have the potential to revolutionise how we collect and interpret such information.
As a simple example, it’s already possible to take a video of an ad or visual concept from a client, show it online in 10 countries without needing to worry about questionnaire design, translation or coding and then automatically analyse the results and generate a fast report comparing emotional engagement, types of emotions aroused and which images evoke most emotion. Moreover, if you are still sceptical, I’d point out that nViso is not the only company working to make various kinds of direct, scientifically objective measurement of emotion more accessible and useable. Whatever we as researchers may think, this kind of data-collection and analysis is going to become increasingly popular, and survey research will have to adapt.
This could perhaps alarm market researchers. After all, if an ad agency or client can take their marketing material to a technology company and get back the kinds of outputs I’m talking about, why bother with researchers? This comes back to my point at the beginning of this article. While it’s perfectly possible to get a huge amount of information about your ad, packaging or product concept by simply “videoing faces”, you’ll get a lot more by embedding the facial imaging into a wider online survey and analysing it in depth. The opportunity inherent in comparing more accurately assessed emotional reaction to brands and marketing stimuli with behavioural and demographic information is huge.
But the value of this kind of “embedded” measure of emotional response is not simply about outputs. Our questionnaires are just too long and too boring. What we really want is a way to get rid of those messy “feeling scales” and nasty ranking and preference measures that bulk up our questionnaires and all too often only yield soft and ambiguous results. If we can embed passive collection measures that let us collect hard data on the soft emotions, we can focus more of our surveys on getting more accurate information on what people are actually doing in their day-to-day lives. Surveys can be shorter, easier to answer and yet at the same time can start to provide much clearer answers to key questions on the real links between emotions and behaviour.
So, while I’m excited about the way facial imaging can help assess the emotional content of an ad or movie, this to me seems to represent only the tip of the iceberg in the use of such methods. The potential to provide a better way to integrate and enhance survey data is, to me, the real reason that approaches like facial imaging are important in research – they represent tools that potentially take us a lot closer to the holy grail of a synthesised, holistic explanation of why people buy stuff and why some ads engage and others don’t. And that, more than anything else, is what we are supposed to be doing. Rather than spelling the end for the survey then, scientific approaches like facial imaging could provide just the kind of support that survey research requires in order to remain relevant in the new age of marketing.
Alastair Gordon is managing partner at Gordon & McCallum
We would like to thank the LINK Institute for running the highlighted study within the LINK Internet Panel and La Mobilière, Switzerland for providing the advertising material utilised in the study.
