By Frank Buckler
Our series ‘Congress Countdown’ looks forward to ESOMAR Congress 2016 by giving you an insight into some of the presentation topics on the programme that will be sure to #WOW you!
In our talk “Sound Research” we try to bring light into the hidden side of market research data. I would like to get your view on the topic!
Why can a product, which senior customers buy more frequently, be in truth more attractive to young people? Why might a factor that is useless in predictive analytics be your most important reason for success? Why might providing some product samples be beneficial but their extensive use can harm sales?
There is a hidden side of business facts as we instinctively misinterpret facts as insights and correlation as causation. It is the first thing taught in basic statistic courses and it is the first forgotten (Thomas Sowell).
As a consequence, most fact-based business decisions as well as most market researchers’ fact-based recommendations are still based in potentially spurious correlations – which means that they are potentially dangerous.
In our talk “Sound Research” at ESOMAR 2016 in New Orleans we will show an example from a case study: There is a high correlation between people’s evaluations of customer service and purchase intention – good evaluations increase purchase intention one could argue. Unfortunately, this correlation turned out to be spurious. Those who own the brand show both better evaluation of the customer service and increased purchase intention. The third, hidden variable “owns the brand” causes both variables to correlate.
“There is nothing more deceptive than an obvious fact” (Sherlock Holmes). Comparing facts and interpreting correlations is dangerous, because in many cases it leads to wrong conclusions and decisions.
Of course, there is a statistical technique called “Regression” that allows for controlling of third variables and thus has the potential to reveal spurious correlations. Regression provides the possibility to learn the unique contribution from each of many different variables towards an outcome variable.
Regression is mostly used when researching for key drivers of success or when quantifying ROI of marketing instruments (MMM). Did you know that even this procedure, despite its wide use, is an overly simplistic approach?
A simple example: A recent Marketing Mix Modeling project revealed that the impact of TV spending of a mobile carrier was unreasonably low. Was there an impact? One would say no because the regression coefficient of TV spending was small. However, more advanced analyses revealed that TV spending caused people to Google and to click on Adword links which in turn drove sales. The variable “Adword spends” suddenly explained the main share of the target outcome. It is not because TV has little impact, it is because the regression approaches only measure direct and not indirect impacts. And this is the reason why neither regression, nor econometric modeling, nor predictive analytics will reveal the true impact of causes.
“If your only tool is a hammer, every problem looks like a nail.” (Abraham Maslow) Regression does predictions. It also can measure direct effects, but not an overall effect which includes indirect effects. Thus, regression analysis often does not tell you what causes success or failure.
There is a third reason that hinders market researchers from understanding how success drivers are related to outcomes:
In a recent study (drivers of pharmaceutical sales) we found that providing product samples to physicians is a good thing – so far nothing surprising. What was surprising was that providing too many samples will do more harm than good. Giving physicians too much samples will force them to substitute prescriptions. At some point, substitution is too high.
This phenomenon is called nonlinearity. We all know about it. Most neglect nonlinearity, because most of the time it is hard to form nonlinear a priori hypotheses. Still, conventional methods force us to do just that.
In the case study we will present at ESOMAR 2016 there is a success factor that turns out to be very important, but only when you allow for nonlinearities. We found that when selling audio speaker systems, the brand should communicate itself with a “relaxed” tone and emotional context. It turned out to be very important that the brand is perceived as “somewhat relaxed”, but not “very relaxed”, which has actually a negative impact.
This example shows why too simple approaches like linear regression regularly fail to understand the true nature of decision processes. This is also why standard regression techniques typically show unsatisfactory explanation power and validity.
And there is a final reason. Conventional approaches like regression always assume success factors to be independent from each other. Too often, they are not. For instance, we found that rebates would have little effect when the sales staffs’ skills are already top notch. Sales skills and rebate level interact or moderate each other in its effect towards sales. The impact of one factor depends on the other.
Those phenomenons are called interactions and are even harder to hypothesize upfront, still conventional methods force us to do just that.
“The pure and simple truth is rarely pure and never simple.” (Oskar Wild) Valuable analytical methods do not just test hypothesis. They have the capacity to explore unexpected insights – no matter if they are nonlinear or moderated.
A core duty of market researcher is to find key drivers of market success. To do just that you need to:
- Refrain from simple bivariate correlations and build on multivariate statistics,
- Refrain from regression and build on cause-effect path modeling approaches and
- Refrain from hypothesis-based statistics and build on self-learning systems that model reality as it is – not as we think it is.
A recent AMA survey showed that only 3% of market researcher are acting on those insights. In our talk at ESOMAR 2016 we will show-case how the Universal Structure Modeling approach help retrieving such valuable insights at SONOS.
As a presenter, I would be very keen to hear your opinion on the topic. I would be also interested to hear on what our talk should focus on and which point should be illustrated more clearly. Thank you in advance for your feedback!
Frank Buckler,
Founder & CEO at Success Drivers
Frank will be delivering his presentation entitled “How SONOS boosted its growth trajectory by leveraging a universal structure modeling” on the ESOMAR Congress on Tuesday 20th September.
About The Author: Frank Buckler, PhD. is founder and CEO of Success Drivers and a distinguished expert in uncovering management success drivers. He is the inventor of the causal analytics platform NEUSREL that eliminated the drawbacks of conventional methods in order to meet the requirements of business applications. Frank has eight years’ experience as a marketing & sales director and is bestselling book author, publishes in magazines and journals, and speaks regularly at leading industry conventions.
