By Carolyn Wu
As technology develops, using traditional methodologies such as face-to-face or telephone interviewing face frequent challenges.
After almost every major presentation that involves a traditional methodology, you receive feedback such as:
- “Why are we still using telephone surveys? I haven’t picked up my landline for a long time!”
- “Can the samples you collect reflect real consumers? It feels our targeted young people are using their smartphones all day long – can they actually reply to your long questionnaire?”
It’s very sensible to have these comments…..
Imagine you are a brand manager – you need to evaluate your brand campaign and advertising effectiveness that targets young people in urban areas. You would expect that through years of building brand awareness and trial, the brand is supposed to show positive performance. But why do tracking results remains static? You would then face challenges from finance to review budgets due to the tracking study results, which you believe to be biased by the traditional methodology. After all, you’re targeting people who don’t pick up phone calls anymore on either landlines or mobiles – they only interact through typing/texting.
Such feedback actually pushed our Insights team to reflect: what if we run a pilot of the tracking survey online on top of the more than 10 years’ telephone survey data to see how the world has changed!
Things started to get a bit tricky but interesting when we started exploring the possibility of conducting a tracking survey online. That’s when we figured out that online panel and online methodologies can be quite hard to adopt in a mid/small size country like Taiwan with a population of 2.3million.
We have learnt that the majority of markets that adopt an online tracking methodology are large markets like Japan – where the large market size allows you more online panel providers to choose from. This allows you access to clearer profiles, better estimates of completion rates and access to more consumers via panel databases.
In Taiwan’s case, we found the opposite scenario to exist. There are only around 3-4 established panels to choose from. Among those panels, the consumer profile is a mystery – one of them was converted from a shopper loyalty point collection program where consumers may have purposely left wrong personal information to avoid disturbing junk advertising emails. Others are built by local venders with little information to understand their panel member’s quality in advance.
The reason international research houses do not build online panels in small markets is usually because of 1) smaller size of prize and 2) more conservative market circumstances where most business are established and the need to change is low. Conversely, in new and booming markets, businesses are more willing to pursue more innovative and efficient ways to conduct research to reflect the dynamic and changing market environment.
In the case of Taiwan, agencies rely on their past experience from other clients to make estimates of completion and screen out rates and recommend combining two panels to complete large samples.
After sorting out the panel providers, some more issues to bear in mind are:
Data Collaboration: Collaboration of data usually requires 2 waves’ parallel running of current & online methodologies, covering a longer period, for instance, half a year to a full year, depending on sample size to avoid data fluctuation caused by seasonal effects. However, running more paralleled waves causes cost increases.
Explanation of Data Deviation: This requires research professionals who know your company and industry very well, as the methodology change may cause data fluctuation, so ensure you don’t use a brand new agency for the online tracking during the transition period. Furthermore, it will be even harder to find the cause of data fluctuation if you operate in smaller markets where the combination of 2 or more panel providers is required.
Price Sensitive Respondents: If panel providers are built based on point collection incentives or small incentive rebate mechanisms, participants are usually quite price sensitive and thus, tend to lead to more VFM driven participation.
By Carolyn Wu, Regional Insights Manager of a FTSE 100 Company
