The Evolution of Market Research

Michael Francesco Alioto

It is clear that the marketing research discipline is changing faster than ever before.  We have experienced more change in the last five years than the previous 70 years combined. But more important than the rate of change is the nature of the change itself.  While the evolutionary change of the past was much more linear and primarily associated with methods and data collection techniques, the current patterns indicate a shift based on a truer “qualitative” evolution.

Phyletic Gradualism or Punctuated Equilibrium?
Much of the marketing research literature[i] and countless industry papers have emphasised the current state of the discipline as being in “flux.”  As with any field, change will occur routinely as the elements of time, space and randomness, as well as systematic fluctuations within it are change-oriented by both nature and definition.  But it appears that the current state of change is “qualitatively” different from the past.

Many industry insiders have viewed past change as a linear set of steady movements within a single species or family[ii] derived from a common ancestor and with a common designated future.  Borrowing from macro-evolutionary biology, this type of change represents the theory of phyletic gradualism associated with Darwin and Neo-Darwinists.  For this type of thinking, one can view change as a steady state with incremental and linear movement occurring based on equal time and space increments.  The future can easily be predicted from the past.

Marketing research had a common ancestor in political polling of the 1930’s. From there it embraced the scientific method from the natural and social sciences and then evolved on a gradual basis.  As a consequence, qualitative and quantitative approaches separated into two distinct species.  Later, the onset of various data collection methods further drove the evolutionary movement in a continuous linear direction.

The recent series of changes provides clear evidence that this proposed gradual and linear evolutionary variation is flawed, as the current change indicates that there is something much more critical and non-linear happening within the field. The seminal work by paleobiologists Stephen Jay Gould and Niles Eldredge sheds some light on change occurring within the research discipline.  Gould and Eldredge proposed that evolutionary change is not gradual and does not occur on a continuous basis but rather transpires in sharp fits and starts.  Species experience long periods of relatively little change in a relatively stable environment.  This is interrupted by rapid change which occurs in a relatively short timeframe.  The species will change or evolve quite quickly and a new species, or even a family, will emerge having very little resemblance to its common ancestor.

During the first 70 years of the research industry, this pattern of punctuated equilibria was driven mostly by changes in the modes of data collection and methodological design.  The research process which replicates the scientific method from the natural or social science disciplines was held constant by the rules, laws, and traditions associated with the philosophy of science.  Evolutionary change occurred mostly within methods and data collection.

The Current Situation: Continued Evolution or Paradigm Shift
If the theory of punctuated equilibrium is valid, then why is there a renewed focus on change in the discipline as if it will radically alter its direction and how we conduct our research projects?

We contend that the renewed focus on disruptive change in the industry is indeed valid, but it is also a qualitatively different change pattern driven by the onset of a significant paradigm shift in both the industry specifically and the environment generally.  Whereas the evolutionary change of the old regime[iii] was driven by changes to data collection and methods, the change being experienced after the paradigm shift is being driven by change concerning the respondents and their universe.  Based on the works of Kuhn and his application of paradigm shift in the natural sciences, a true shift is an infrequent occurrence driven by an inability of the old paradigm to solve puzzles or questions.  Compensating for fundamental change within the respondent body and its surrounding environment is something that the old paradigm is unable to account for currently.

The old regime was dominated by a punctuated equilibria evolutionary structure based primarily on changes associated with methods and data collection procedures.  The current paradigm shift is being driven by critical characteristics of our respondents and includes a basic understanding of humans and their attitudes, as well as the insights which we glean from the very marketing research studies we conduct.

A “New World”: Key Attributes of the Post-Paradigm Shift Marketing Research Discipline
Based on the current change process and a focus on the human subjects that we are studying, the new attributes identified by the current paradigm shift can be segmented into five major categories.  The dramatic outcome of the paradigm shift is the move from focusing on findings and resultant behaviors concerning a particular product, service, or solution to a more complete comprehension of the respondent’s decision intelligence or decision making process.  Focus has shifted from decision-making to decision intelligence[iv].

This represents a major shift in thinking and assessment concerning the way we approach and internalize the marketing research process. 

The first category (1) addresses the need to understand human nature in a total sense of the term.  Humans and their decisions were viewed as being rational by definition or based on a-priori reasoning.  There was little or no room for irrational or non-rational thought patterns or behaviors.  Theories and general normative viewpoints from psychology, anthropology, among other disciplines have essentially been ignored.  The rise of behavioral economics has proven to us how misguided we were.  Seminal work by Brainjuicer[v] has proven the importance of the non-rational within basic decision structures.  It is clear that we must understand and comprehend the respondents in their totality and assess the contributions that both rational and irrational thoughts and behaviors have in our studies.

The second category (2) addresses the environment in which humans function and make their various decisions.  Historically, we studied human decisions and nature, either void of environmental factors, or at a single plane, whether global, regional, or individual.  Based on the work by Kenneth Waltz[vi] in international politics, we know that the three levels of analysis tend to interact with one another—humans are actually influenced by one or more levels simultaneously.  Too often marketing researchers constrain their study design and analysis to their human subjects within a single level and profess that humans do not interact or move between levels.  As globalization has become apparent, we recognize that respondents are greatly influenced by all three levels of analysis, creating hybrid influences on behaviors and the decision-making process.

Similar to the old regime, technology is continuously influencing and setting the foundation for change in both methods and collection modes.  This is our third category (3) under the new paradigm.  Within the old model, method and data collection change was seen as linear and a conduit to gather consumer information.  In the new paradigm, it will be influenced not only by technology and technological breakthroughs, but also by the interaction with humans themselves.  One can expect the “death” of ordinary surveys and traditional focus groups which simulate more traditional experimental or quasi-experimental designs or studies.  From this outcome, we will observe the rise of gamification, in which the questions and functions of techniques will be disguised to more closely resemble the (normal) functioning and actions of the human brain.  Additional information will be collected through social media in a more “discussion-like” format.  This will stimulate natural human dialogue where more biological-based measurements will be triangulated and compared with attitudes and statements.

The post-paradigm shift world of marketing research will also need to view business problems and objectives with a broader lens, chiefly at the industry or enterprise level.  Much has been written about the rise of big data and predictive analytics, which is our fourth category (4).  Under the old regime, much of our research was focused on a single problem, in a single space, at a single point in time.  While this approach supports a simple strategy, it artificially reduced the complex nature of life and the consumer decision-making process.  The use of big data and predictive analytics provides complex levels of data and is able to assess the research problem and objectives from a number of dimensions more accurately reflecting the complex nature of the problem.  Critical for the success of big data and predictive analytics will be for it to successfully comprehend data across space and time.

Finally, the fifth (5) and last category of the new paradigm world is focused on how insights are socialized throughout the client organizations.  Classical reports and PowerPoint presentations associated with traditional marketing research are obsolete.  Many have called for the death of the PowerPoint presentation.  The onset of marketing research insights and storytelling reveals the future of our industry.  Stakeholders and decision-makers are no longer contented with data or findings.  The new focus in the post-paradigm world will be on insights that are holistic and tell a story of what the customer needs and wants.

The Road Ahead: Implications for the Marketing Research Discipline
Today’s market researchers are much more complex in our understanding of human nature and the dynamics associated with human decision-making and thought processes.  While humans were always complex in their decisions, an openness to learn from other fields, disciplines, and technologies has armed us with the ability to better comprehend the decisions and thinking of our human subjects.  One can expect that change will occur at an ever more rapid rate and the types of changes will be much more meaningful.

The outcome will be even more significant as we will be able to more accurately explain past behaviors and current decisions, but even more important, predict future behaviors and decisions with much more certainty.

Truly, our marketing research discipline is moving toward the prospect of building its collective human intelligence and the results will be much more dramatic.  We live on the edge of forever, and the universe is our new portal upon the future of human understanding.

Michael Francesco Alioto is Vice President, Global Methods, at Gongos Research

[i] See the work by Kaden, Lina, and Prince (eds.) (2012) which is a very good representation of the literature on this type of change in the marketing research discipline.  Of special interest is the first chapter by Lewis and Chadwick which provides a good assessment of the change factors driving the marketing research industry and what researchers should expect within the next few years.

[ii] We will use the terms associated with macro-evolutionary theory and paleobiology to address some of our thoughts throughout this paper.  Common ancestor will be used to refer to a common point or origin for a common event; species is a particular technique or method, and family refers to a major sub-category, such as qualitative or quantitative approaches.

[iii] We will refer to the pre-paradigm shift period as the old regime and the post-paradigm evolutionary shift as the new regime.

[iv] The shift from attitudes to human intelligence is mainly a shift from a single set of attitudes or attributes to a process of understanding humans and how each of their individual attitudes, behaviors, and personal characteristics are viewed within this process.

[v] See Wood, et. al. (2012).  This research agency has done some outstanding work in this field and have run a series of tests which have supported the main findings and ideas of BE in the marketing research realm.

[vi] See Waltz (2001) for a detailed description of the application of this theory to international politics.