Consumer insights in the machine age: Looking for the soul in the data

Consumer research may have digitalised radically over the last few years, insights teams still need to uncover the humans behind the data. We asked insights experts how they connect emotionally to people in the machine age, and what a modern insights team should look like.

When Carol Fitzgerald, President & CEO of BuzzBack, refers to the machine age, she always mentions the 3 A’s impacting the insights industry: Artificial intelligence, Automation and Agile. Everyone’s focused on these, but the definitions and adoption rates vary by client. “Today, many companies are adopting some form of the 3 A’s but in many cases, they are ‘plopping’ the technology in their organisations, tasking their teams with technology and losing sight of the ‘insight’ or ‘story’.” The role of insights needs to be in curating the bigger picture story around the data, Fitzgerald believes. “The big data, is in fact, BIG. Right now, many of the insights’ folks are mired in the data. Their role should be in providing strategic consultation about where to take the data and how to derive the true insight.”

Gut feeling

In the machine age, an insights team needs to have advanced analytics and machine learning skills, says Catherine Willis. As a Global Consumer Insight Leader who used to work for Delta Air Lines and LG, Willis feels it is no longer sufficient to just have traditional market research skills. “There needs to be a balance of traditional researchers and analytics professionals. Through collaboration across those disciplines, deeper insights are unlocked.” In addition, she stresses the importance for these disciplines to be on the same team. “After all, the real power today is how the dots are connected across the disciplines to create a full picture of the customer, which is needed to create truly customer-centric organisations.” 

Niels Schillewaert, Managing Partner at Insites Consulting, believes that the added value of machine learning is in ‘Intelligence Amplification’ – IA rather than AI. “Machine learning works well in structured settings where all relevant information can be studied and outcomes anticipated for clearly defined goals, but it works less well in situations where goals and inputs are not well defined, and where empathy, emotion and gut feeling are key.” It is in those situations, Schillewaert stresses, that humans are needed to interpret data and translate them into impactful insights. A benefit of this is that the value of ‘judgment’ tasks will increase as the cost of ‘prediction’ and ‘codification’ decreases due to machine learning. “Therefore, indeed, the core of what market research and insights professionals should focus on is understanding human motivations and behaviour.”

Connecting the dots

So what should today’s insights team look like? Data scientists are key to the interpretation of data and use of algorithms, and as such are an integral part of any team, says Willis. “So, with head count often hard to come by, at least on the corporate side, insights teams will need to rely on market research vendors more for primary research and less on DIY tools.” These tools, despite many advances, are still highly labour intensive. So, to incorporate data scientists and analytics experts alongside traditional insights professionals, there does need to be some outsourcing, according to Willis. “Since most data science relies upon a company’s data, for security purposes, that should be inhouse. Experienced insights leaders who can connect the dots and think strategically are critical alongside data scientists with the latest skills and expertise.”

An insights department needs staff who can steer machine learning (i.e. data engineers and analysts) which can provide high quality models and knowledge based on valid and quality data, explains Schillewaert. “To do this well though, data analysts need to be assisted by data translators to generate actionable insights. These people need to be able to speak the language of the C-suite. This requires managerial skills, developing business expertise, and action skills such as storytelling and creativity. The problem is that many insights functions today still underestimate the importance of data analytics.”

Primary research provides the why and some of the how, while customer data provides the what and some how. You need both to truly understand the customer decision journey.

Huge gap

For BuzzBack staff and resources in the current environment it becomes important to parallel the role of insights teams at its clients, explains Fitzgerald. “That means, we too, need strategic ‘consultants’ to help clients fast-track their data. We can implement and deliver data, but it needs to be couched in what the story is and how clients can activate and socialize their insight faster and easier. We can bridge companies’ use of algorithms and machine learning, augmenting their efforts – especially because their resources are shrinking.”

In addition to leveraging the technology for automation and AI, Fitzgerald signals the need to integrate the consumer experience more. “To keep the human touch, we need human centricity and understand them as people first. And we need to meet consumers where they are going, not just where they are today.” As an example, she says that consumers spend up to 80% of their time on mobile devices, yet less than half of research is conducted this way. “This is a huge gap in how we connect and relate to them. There are very simple approaches to integrating mobile and other behavioural techniques to algorithms and machine learning. Our team needs to reflect both.”

People first

Fitzgerald sees behavioural economics as an important trend or overlay across many insights and research teams. “Those companies adopting behavioural economics are increasingly deploying observational and other qualitative methods to explore the consumer journey; from how they shop to their needs and even their unmet or unarticulated needs.” She explains how BuzzBack clients couple these kinds of observational methods with other methods by integrating proprietary projectives to elicit emotional and implicit response. For example, a patient journey study might include ethnography, but also a qual-quant exploratory with eCollage where patients map their experiences and explain why they feel as they do at different parts of the journey. “These kinds of techniques further help to dimensionalise the journey and bring to life the experience.”

It all comes down to putting people first, Schillewaert thinks, because understanding ‘why’ matters. “Technology does not solve problems, in the end people do. Big data makes us data rich, machine learning makes us pattern rich, but we are often still insights poor. It does not work in isolation.” Which is why, at InSites Consulting, they look for a structural consumer collaboration through communities. They provide clients’ data and marketing models with a ‘soul’, as Schillewaert describes it. “The upside of communities also goes beyond the research aspect: it allows senior management to actually connect to people. For me, the key to understanding the consumer decision journey and tell the story behind the numbers is in triangulation with rich and immersive primary data-collection methods.”

The core of what market research and insights professionals should focus on is understanding human motivations and behaviour.

Sweet spot

Indeed, primary research is still critically important, says Willis. She stresses that customer data alone, no matter how sophisticated, cannot replicate talking to the customer, observing them in their environment and digging deeper with projective techniques and survey structures. “Primary research provides the why and some of the how, while customer data provides the what and some how. You need both to truly understand the customer decision journey. If only analytics are used, only half the story is known and the risk of making very wrong decisions is high.” The sweet spot, she feels, lies in strong primary research alongside data analytics to understand behaviour and trends. “An organisation cannot consider itself truly customer-centric without insights professionals who can connect the dots and be an objective voice of the customer.”

According to Fitzgerald the documents today’s largest consumer companies present to investors show a focus on consumer centricity, product and services innovation, and the leveraging of ecommerce/direct to consumer in a changing retail landscape. “Interestingly, consumer-centricity is the antithesis of machine learning and the 3 A’s – there is tremendous tension in filtering the big data and exploring the changing consumer and what makes him/her tick. A focus on consumer centricity means understanding who they are and what motivates them; getting deep, exploring emotions that drive behaviour. While the research sector is evolving and transforming every day, it is still racing to catch up with where consumers are going and how they’re communicating.”