Is Machine Learning a Threat to Market Research?

By Sinead Hasson

Machine learning is a new field in algorithmic computer science which enables powerful software to ‘learn’ without being explicitly programmed. The software platforms interrogate data stores in search of patterns and, depending on what they discover, then define further lines of investigation all independent of human direction or intervention. The end result? An ‘intelligent’ infrastructure that has the ability to automatically generate highly accurate data forecasts, and make operationally sound decisions, too.

Machine learning evolved from the study of pattern recognition and computational learning theory in artificial intelligence and has recently been gaining momentum in a wide variety of industries, including market research. It is typically used to identify opportunities to raise efficiency by finding patterns in data that humans are unlikely to uncover. The problem that machine learning claims to address is one of bias. Traditional researchers use their human-touch to investigate and interpret data they have collected that is based on a hypothesis or a specific research brief. As a result, conventional researchers approach their data with a predetermined idea of how it should be interrogated and what they are looking to find. In this way, the research parameters have already been defined, possibly at the expense of additional useful insights that because they remain outside of those parameters, will go unnoticed. In other words, they don’t always know the right questions to ask in order to maximise value from a data pool in the first place.

Machine learning, on the other hand, doesn’t come with an agenda. Moreover, it can also sift through far more data than a human analyst is able to, and trace the gaps in the process of enquiry based on the trends it has already identified. It then has the ability to discover patterns, produce forecasts and subsequently influence decisions. Almost all decisions made by major commercial organisations are operational in nature so the potential for machine learning to raise corporate efficiency is huge.

So, not only is this technology able to examine petabytes without being directed, define the right questions and identify more trends than a human researcher can, it can also produce reliable forecasts and as a result make better judgements than us too. With all this in mind, does machine learning spell the end for market research professionals?

Thankfully not. Machine learning lacks one invaluable quality: it isn’t human. Fortunately for us, each researcher possesses something unique; they’re own bank of experience upon which to base their interpretation of results. A human analyst’s innate bias can actually be a good thing, even if it does narrow the search parameters. Not all insights can be gained from logic alone and we don’t live in a world governed by computers. In our uniquely human world we need researchers with the ability to take into account social, economic, historical and cultural situations – something that a computer is unlikely ever to do as well as we can. Given that much of the interpretation of quantitative data depends on qualitative experiences, statistical and mathematical approaches won’t always produce the best results. Possessing and understanding the nuances of human motivation, desires, rewards, emotions, humour and sometimes even egos equips us with interpretive skills that ‘the machines’ will never have.

Like deep learning and data mining, machine learning is just another tool for researchers to use and, as with any new technique, time must be taken to study the practice. There are a huge variety of algorithms emerging, ranging from the comparatively simple to the highly complex, and it’s important to know when and how to use them. The field itself remains somewhat academic too, and is growing the fastest in sectors with the investment capability to incubate teams of data scientists, such as retail and investment banks.

This won’t last, however. Platforms are already starting to emerge that aim to deliver the power of machine learning in a user friendly manner, so market research professionals would be well advised to familiarize themselves with this emerging field, since its mainstream market research application may be closer than it currently seems.

Is it a threat? No. Machine learning lays out the bigger picture, in fact it lays out the biggest and the broadest picture possible, but it is our human qualities that will allow market research professionals to continue to inform the strategic decisions that really drive change, independent from an operational analysis or a statistical forecast.

Sinead Hasson, Managing Director, Hasson Associates

Sinead founded Hasson Associates in 2008 and now has more than 17 years of experience gained in roles with Pricejamieson and RPCushing. Hasson Associates is a niche consultancy specialising in placing marketing and market research/insight professionals in research agencies, brand and strategic marketing consultancies, market analysis and business information providers as well as in-house research teams in the UK and internationally.