Carpe Diem: How to embrace tech disruption in the market research industry

The research industry is notorious for being slow to adapt. However, right now we seem to have no choice as the influx of technology is beginning to force disruption onto us. As research departments integrate into broader insights functions and companies start separate data analytics teams, the role of the researcher is shifting. Companies have access to more and more data and the technology to analyse it in real time. It should indicate that the traditional skills of the market researcher – creating stories and finding meaning within data – mean that we have the opportunity to be leading the charge within our businesses.

Simultaneously, technology has developed so that many of our time-consuming tasks, from data coding, text analysis, sample selection, and management, as well as questionnaire creation, can now be done at the touch of a button. So we have the opportunity and the tools to lead the charge.

With the advent of self-serve and automated tools, it has become practical for anyone to get answers to their basic business questions right away. Looking towards the future we see lines blurring across the industry. Research will not be owned exclusively by insights departments; our friends in marketing departments will start to run their own (automated) tests, hopefully after their insight colleagues have approved the test methodology. This raises the question: what will researchers do, become key strategic partners or be made redundant? What should we all do!

In the immediate future, as technology and automation continue to disrupt the status quo, we have to recognise that research now needs to defend, advocate and promote the science that is the foundation of what we do. As insights become more accessible it is important to acknowledge that just because anyone can conduct their own project, draw conclusions and analyse the data, it doesn’t mean that everyone should. Using the wrong question, sample or analysis can make the “insights” incredibly misleading. Keeping the world of business educated on the science, and not just the tech, behind what we do, as well as the differences between methodologies, is crucial as these technologies spread.

Five trends

Looking towards the longer term there are five major trends we’ve identified in the market research and technology space that present us with the opportunity to become true strategic partners to our businesses. I anticipate the following:

1. Continuous push to agile. As business becomes more agile the demand for iterative insights will increase. This demand will either be filled by us or by other forms of data such as social media or behavioral. We need to focus on this space.
2. Insights solutions will get more flexible and modular in order to test ideas in the right context and at the right speed. This means that more people will be able to do the research that they need and researchers will need to play a strong training role to ensure quality and actionability remain.
3. More sophisticated data analytics will mean an end to throw away data. Approaches such as machine learning will mean that data has the potential to be used for much longer and have a more powerful impact on business. Researchers will need to curate these analyses. This blurring of insights and data science needs to happen quickly.
4. With more people accessing insights, lines between qual and quant will blur, especially as businesses strive to make every function data driven. With text analytics allowing qualitative at scale, we researchers will be able to impact more areas of the business faster than ever before.
5. Most importantly, we will take the first major strides to move from insights to foresight by having data from various places connect and form real-time calculations that will help us to predict outcomes. We need to ensure we have these data science skills in our teams and not in a separate department called data analytics!

It’s time for the industry to stop kicking the can down the road, and embrace the opportunities that new technology and modern methods can offer. We must start by defending and promoting the science behind our methodologies but then embrace the new opportunities automation and particularly data science are providing for us. If we don’t, there are now several other departments in businesses that will.

It’s a great time to be a market researcher: data, insight and consumer centricity have never been more important. It’s up to us to seize the day.