Rooting for marketing research

Asking “how can we retain our academic roots and yet be more commercially savvy,” suggests a tension—or even contradiction—between these objectives. The Marketing Science Institute (www.msi.org) was founded decades ago on the premise that this need not be the case, that in the words of a former academic Executive Director, there are “thoughtful practitioners and practical academics” who, given the proper forum, have much to learn from each other. Granting this, we still need to “ensure that academic-based researchers speak the language of business and marketing.” Drawing on my background in marketing research and MSI’s experience, here are some thoughts about how this might be done.

First, the language of business boils down to one word: profit. Assisted by research, marketing’s role is to achieve sustainably profitable growth by helping managers make the best use of available resources. In effect, marketing is a form of risk management, minimizing the likelihood and consequences of making bad decisions. More positively, marketing creates “real options”—greater degrees of freedom for future actions such as launching new products or expanding into adjacent markets. Thus, we begin to create common ground (and someday, a level playing field?) for discussions with CFOs and CEOs about the value of marketing as an investment, not an expense.

What practical contributions can marketing researchers make to this discussion while maintaining the rigorous standards observed in academic research? One answer may be found in the shift from a deterministic to a probabilistic mindset in conducting research and communicating findings. Years of doing research and consulting for telecom and energy utilities gave me great respect for engineers and the accomplishments of their linear style of problem solving. This way of thinking also proved a barrier, however, to their appreciation of the proper role of marketing. If your only hammer is a fixed input-output model, you are likely to see the world as so many pre-existing markets ready to be pounded.  More typically, marketing creates rather than “finds” markets (as any good FMCG marketer could have told you at the beginning of the 20^th Century). Fortunately, researchers today have a much larger box of tools that are better adapted to the role of marketing as risk management in volatile environments.

For example, Ron Berman (Wharton), Elea Feit (Drexel University) and colleagues demonstrate how Bayesian modeling incorporating “priors” from previous research can be used to reduce sample sizes needed to detect statistically significant results, accelerating interventions without compromising validity. Analyzing over 2000 A/B testing experiments conducted on an online testing site, the authors find that more than half are “p-hacked”—that is, the experiment stops when a significant effect is detected, inflating false discovery rates and leading to bad decisions. They develop instead a “test and roll” methodology to quantify the options researchers face to avoid p-hacking while maximizing the speed and effectiveness of marketing interventions. In contrast to the traditional approach to A/B experiments as hypothesis testing, the authors “re-frame them as an explicit trade-off between the opportunity cost of the test (where some customers receive a sub-optimal treatment) and the potential losses associated with deploying a sub-optimal treatment to the remainder of the population.”  More consumers receive optimum marketing interventions sooner (and often at lower costs), thus fulfilling marketing research’s role in facilitating profitable growth through efficient and effective resource allocation.

In addition to Bayesian modeling, artificial intelligence and machine learning algorithms are examples of the more “open” and flexible approach to marketing research consistent with creating rather than “finding” markets. Are there corresponding lessons to be learned from academia that can facilitate communicating research findings in equally “open” and flexible ways?  Here we can complement the latest data science techniques with longstanding findings from social science and the humanities.

In short, human beings are story-telling animals. From an early age, we understand and respond to stories, not in the closed, stimulus-response terms of rat psychology but as windows on our world—framed from particular points of view, but subject to re-interpretation on the basis of subsequent developments. Narratives (and conversation in general) are structured by abductive reasoning—extrapolating from past experience to possible futures, paralleling the “what if” scenarios and policy experiments that can be conducted with data science models. As such, storytelling is the most “natural” and effective way to convey generative insights to marketers, informing rather than strictly dictating their decisions. When deciding how best to “manage [the] two-way knowledge transfer” between academics and marketing research professionals and “which classically ‘non-research skills” to develop, we should focus here.

Sources: Ron Berman, Leonid Pekelis (OpenDoor), Aisling Scott (Facebook) and Christophe Van den Bulte (Wharton), “p-Hacking and False Discovery in A/B Testing.” MSI Report No. 18-130-10.  Ron Berman and Elea McDonnell Feit, “Enhancing Power of Marketing Experiments Using Observational Data.” MSI Report No. 18-116-06.