While definitions of quality in market research can be complex, there’s one aspect of poor quality that is immutable: Anyone who lies on a survey – or in any interaction as a panel member – to get a reward or make mischief, has no place in a sample. This behavior is, at its core, fraud.
Estimates on the frequency of fraud in market research vary, from the low teens to as high as 30-40 percent. No matter the frequency of fraud, however, its effect on data and insights can be disastrous, leading to flawed insights and poor decision-making. Simply put: If you can’t trust that your audience is truly who they say they are, then you can’t trust your results.
And as long as we offer rewards for taking part in research, the temptation for some to engage in fraud will exist. Yet there are steps we can take to educate and protect ourselves as market researchers, leveraging practices and tools that ensure the integrity of the audience and their responses, and therefore the insights we gain, that power decision-making and fuel growth.
Market research fraud is becoming more sophisticated
Digital fraud has increased across all industries over the past 18 months, including our own. Perhaps this is due to more people being home with more time on their hands, or more available technology to aid in cheating. Perhaps surprisingly, brands in retail and other sectors have even come to Dynata for advice on fraud prevention among their own customers.
Identifying fraud
It’s important to separate fraud from quality issues caused by inattention. Straight lining, poor open ends, contradictory data or answers that are not credible are often caused by fatigue, confusion, frustration, distraction or hesitance to share. While the effects we see in the data can be the same, whether people’s motivations are good or bad, but the prevention methods in each case are very different.
Impact on the data quality
While the goal is to prevent all fraud, it’s possible for a sample to suffer from a low level of fraud (in the low single digits) without impacting data. The greatest risk of fraud is not the occasional scoundrel, but an organized attack on an individual panel, source or survey. Another concern is the impact of fraud on low-incidence studies. Bad actors work to qualify for any remuneration. For example, if 1% of a source is fraudulent their presence on a 5% incidence study could be high if they continue to qualify while honest participants screen out.
Dynata’s fraud prevention & management goals
Dynata’s goals in fraud prevention and management represent the balance between the experience of our respondents – the people taking our surveys – and the research and business needs of our clients in knowing they are hearing from real, engaged and qualified people.
For Dynata, fraud prevention and management is a two-part process. It starts with collecting clues about who an individual is, gathering the data both comprehensively and quickly.
These clues include, but are not limited to:
- Device data
- Geolocation data
- Third party matches against known lists
- Behavioral data in our own platform
- Profile and join form data
Once that data has been collected, models determine whether a respondent is real, and unique, based those available clues. This creates a positively reinforcing cycle as we collect more data, machine-learning based models make decisions more quickly and accurately.
In addition to technical fraud prevention, we are committed to creating a more transparent and worthwhile experience for the consumers sharing their data with us. This includes:
- Relevant surveys that don’t waste their time.
- Reward and loyalty offerings that match the value of the data they give us.
- Security and protection of their information to ensure it isn’t misused or comprised.
That commitment extends to targeting as well, making sure that our clients’ chosen audience is vetted and verified – both using industry-standard tools and our own proprietary processes and solutions – so that we can identify, and remove, fraudsters before they can be included in any sample.
Finally, we monitor performance and behavior within the survey to detect any early-warning signs or activity that indicate suspicious behavior leading to an attack, intervening and correcting before that action has a chance to be successful.
In addition, we work collaboratively with our clients to help identify and implement tools to prevent fraud on their side, protecting the integrity of the data and insights they receive to help ensure decision-making isn’t compromised in any way.
Dynata’s strategy: Innovation and expertise
Given the increasing level of sophistication among fraudsters, Dynata’s fraud prevention and management strategy embraces innovation – of tools, processes and solutions – to keep pace with the threat, leveraging multiple fraud controls at every touchpoint with every person on our panels. It’s innovation that combines advanced technology like AI and machine learning with manual review and other expert practices drawing on our 40+ years expertise, as well as the adoption and use of both general purpose and industry-specific third-party tools.
Increasingly, we use AI & machine learning to create norms for things like keystroke patterns, answer speed and other response characteristics, then use those benchmarks to examine outliers and take fast and early action.
Fraud: The fight for the future
The opportunity for fraud will continue to grow both within our industry and outside it. And its impact on quality – for data, research and insights – will continue to be felt if not properly addressed. But as fraudsters become more sophisticated in their practice, so to will the technologies and techniques available to identify, combat and prevent that fraud in the future.
It is a hygienic practice, maintaining that your panelists are real and vetted and therefore ensuring the data and insights they offer are high quality. And it requires a focus on all aspects of the process, starting with prioritizing participant retention and trust to that lead to better surveys to collect data and valuable insights that are uncovered. Data quality, and the sustainability of online research, requires this hygiene and vigilance.
As Dynata and other companies invest in more sophisticated technology and controls, the emphasis will always be to protect and extend the quality of data, research and insights. We are in a continuous arms race to exclude them requiring continuous, concerted, targeted effort. With more proprietary panelists than any other provider, Dynata has masses of data to act on. The more data we have, the more reliably quality management tools can predict quality and the faster we can prevent fraud.
