By Jonathan Pirc
Increasingly, research companies are pushed to provide smarter solutions with a quicker turnaround and for a lower cost. When it comes to acquiring respondents, the question remains as to whether it is possible to provide good quality research under these conditions.
Of course, whenever a challenge like this arrives, it presents opportunity for innovation. Innovative methodologies are vital in protecting the quality of research, and continuing to justify market research investment to end clients. One area of particular area ripe for innovation is survey sample acquisition methodologies using new platforms.
At Lab42 we began to look at sourcing samples through social media, e-commerce and online gaming sites to find respondents on the platforms they are already using and fully engaged with. We can then offer incentives that people really care about. While we knew the concept had great potential, particularly in the speed of filling quotas, questions still remained; what is the impact on the quality of the research? What effect does the use of such a radically new model have on the research results?
We set out to answer those questions by conducting a large scale ‘Research on Research’ survey. We generated respondent samples using both a traditional panel provider (TPP) and our proprietary methodology (Lab42), ensuring both were representative of US Census data, with quotas set up to ensure the populations were identical to one another. The results showed that quality and content of the results was staggeringly similar across the two methods and, in some ways, were even better with our sample.
The first benefit of social sampling is the speed with which it can fill quotas for particular demographics: the under 35 category filled faster and the 35-55 age groups remained even compared to our traditional panel provider. This might be an expected outcome given the demographics for social media use, but while you might expect those on their favourite sites to take survey-taking less seriously, fewer Lab42 respondents had to be removed because of data quality standards. Not only can social sampling give you data faster but it can potentially give you better data.
When the two samples were asked about their behaviour and attitudes, their responses were overwhelmingly similar across a variety of areas. While you might expect the social media sample to be more interested in the internet and technology, when looking at the top social media channels, the social media usage was the same and about half of the respondents from both samples consider themselves to be early majority adopters of technology. Even their media consumption habits were similar, with both sources likely to get their news and information from television more than online.
This speaks to a greater overall change in the social media landscape. Where once such sites may have been niche, now they command a much wider audience virtually identical to that of the pool of traditional respondents. Perhaps this is best demonstrated by looking at the attitudinal similarities. On everything from international travel (Lab42 at 18% and traditional panel providers at 17%) to visiting a place of worship (Lab42 18% and TPP 19%) to political affiliation (Lab24 33% and TPP 34% identify as Democrats), the samples were virtually identical to one another. As these sites become increasingly mainstream, their potential use as a source of survey respondents has become more viable.
While the two groups were overwhelmingly similar in most ways, there were a few key differences. The social media sample included less professional survey takers: more than half of the traditional sample compared to only 26% of the Lab42 respondents report being on panels and the traditional sample are more likely to be on multiple panels and to have taken multiple surveys in the last month. The social sample are more likely to have shopped for electronics (54% to 39%) and clothing (72% to 39%) in the last month. And in terms of demographics, Lab42 has a better reach among younger generations (millennials, generation Z) and older respondents (55 plus). This means that social sampling can provide better reach to consumers and demographics that are not represented by traditional panel providers: an untapped market that could be hugely beneficially to certain industries.
So what does this mean for the future of acquiring survey respondents? Online panels may never entirely be a thing of the past but, as time goes on, their relevance may diminish as more innovative methodologies are refined. The research shows that sourcing respondents through these new channels is not just a viable option for those looking for a fast turnaround or to reduce investment, but also for those who still need quality results. Additionally, as the results from the sources are so similar, the client can combine sample sources without the risk of influencing the final outcomes of their research projects.
Jonathan Pirc is Founder and VP of Product, Lab42.
1 comment
The way I see it is that every corner of the internet attracts a different kind of person. Some people like to play on facebook and twitter. Other people like to play on forums and review boards. Still other people are big fans of searching with google and bing. We all gravitate to the part of the internet that best speaks to our own personalities. By using sampling methods that focus on reaching many parts of the internet, you have a better chance of generating representative samples. Certainly not probability samples, but definitely representative samples.