When Clayton Christensen published The Innovator’s Dilemma in 1997, it was as if a mist had been lifted and the process and results of disruption had become that much clearer. We could look around us and see what he was talking about in any number of industries – photography and computing being just two of the most prominent at the time.
But just as most politicians missed half of what the economist John Maynard Keynes was talking about in the Great Depression, so many of us missed the other part of what Christensen was saying. Sure, he was talking about the disruption of entire industries but the results he was focusing on affected companies, modes and methods, not the raison d’ȇtre of the industries themselves.
When the steam engine disrupted the clothing industry, it wasn’t to deprive us of clothing – it was to clothe more of us at a cheaper price. When the internal combustion engine came along and horses became redundant, it wasn’t to wipe out personal transport, it was to democratise it. This also applies in the fields of communications (phone, fax, email, social media), the processing and dissemination of information (mainframes, minis, PCs, laptops, tablets) and photography (plates, film, color, digital). In none of these was the core benefit of the industry (talking to each other, processing information or capturing memories) affected. In all of them, it was democratised, made more widely available at a cheaper price.
Democratising insights
And so it is with insights. If we think about it, disruption has been with us since the birth of this industry a century ago. It started off as a highly exclusive, rare, expensive form of consulting between gurus and the captains of industry and then became increasingly industrialised. It went through numerous changes in the mode of data collection – each more disruptive and cheaper than its predecessor. Data processing – once the exclusive purview of the owners of vast mainframes – came to the user’s laptop and now is being automated through the arrival of AI and Machine Learning. New methodologies were introduced – pricing and market mix models, conjoint, semiotics, digital qual and ethnography. You name it – the list goes on and on.
Today we are more aware of disruption because of the sheer speed with which it occurs and the lightning impact of technology. But really we are just keeping up with the pace of change throughout the whole world, whether in our work places or our homes. Sometimes, the effects of this change or disruption can be ugly. We have seen the big goliaths of the industry stagger and many of their smaller, more traditional, brethren collapse altogether. The ESOMAR Global Market Report in recent years has charted the slow but inexorable decline of traditional survey research.
Can we maintain the quality?
In the meantime, we have seen the Venture Capital insights investment boom bring us probably over 100 viable new competitors. As Christensen warned us, many of these would be offering products and services of a demonstrably lower quality than that available from legacy market research. But what he also said was that, almost inevitably, quality would improve and eventually surpass historical norms (again, think photography). The so-called “DIY” data collection platforms are a prime example of this, with the likes of SurveyMonkey now employing serious research design talent to advise clients on best practices. Another example was when a brash young technology entrepreneur got up at a research conference and pronounced, “the problem with you researchers is that you are lousy at technology and the problem with us in technology is that we are lousy at research”. That young man, six years later, is now one of the emerging leaders of our industry.
The central point here is that, however great the disruption, the heart of what we do and why we do it remains the same. We are still in the business of providing great insight based on solid data so that companies, politicians and not-for-profits can make better decisions.
Where we have to be careful, however, is in not confusing initial degradation of quality with an abandonment of core principles, our ethics or our integrity. The data on which we base our insights still needs to be of high quality, accurate and reliable. And our ability to interpret that data and communicate it to those that matter still needs to be first class.
What we cannot do is to confuse disruption with wholesale, unethical destruction. As I write, the owners of Sports Illustrated have fired 50% of their newsroom, replacing them with unpaid bloggers and freelancers – even people who will pay to get their article into the publication. Other parts of the news media around the world are seeing the same sort of degradation of the product based solely on the need to slash costs. That is not disruption. That is destruction. If ever we were to see a similar trend in the insights industry, that would be the time to cry “enough”. If quality, ethics and integrity come under attack, that is when we should mount a counter-attack.
At base, our job today is to embrace disruption, knowing that our core mission remains the same. Change can be good, it brings vibrancy, it challenges us and our ultimate product can be the better for it. But, at the same time, we have to imbue the new with the standards, ethics and integrity that have withstood the tests of time. If we don’t do that, we lose our reason for being.
