Big Data Plus Research Means More Accurate Results

TIM MACER

Yahoo’s Himanshu Shekhar Jha tells Tim Macer how and why

Himanshu Shekhar Jha is head of data solutions at Yahoo! Asia Pacific, where he has been an advocate for blending big data with market research data. Yahoo has long played a pioneering role in developing fundamental big data technologies such as Hadoop and continues to be an innovator in this area.

Jha was educated in India and has worked in Singapore in IT and Data Science. Across a string of senior IT positions, he has been able to use big data not just to predict the near future but to generate new strategic insight. Given the inherent difficulties in trying to bring together market research data and big data, I asked why was he such a strong advocate for it.

“India recently concluded one of the biggest elections on the planet. Market research was in big trouble, because in past elections, the poll results were wrong almost every time. This time, every reputable market research company was on mark, and the reason was that they started using big data together with market research data.”

The polling organisations ran sentiment analysis and built predictive models around both poll data, which arrived infrequently, and sentiment data, which poured in all the time.

“Some models would be run 10 or 20 times in one or two weeks, which meant each of the predictive models became better and better. So when finally it came to the experts to give their opinion, it was a big help to them in getting it right.

“I think psephology has moved to a different level now, and for the first time market researchers are getting the kind of respect they deserve. This is good for society too, because some of the same techniques and tools are being used by the politicians to evaluate development projects.”

Jha provides further examples, suggesting that many corporate brand performance trackers and customer satisfaction trackers cannot make sense in isolation if they do not ‘ingest’ other sources of data such as sentiment analysis from social media or trend data from online searches. “If you look at some of the consumer or luxury brands, or electronics, it’s very difficult to gauge the consumer response or any brand uplift given the frequency with which [campaigns and products] are launched and rolled out in every country. A lot of that has already moved to big data analytics – you have online platforms that provide a daily, weekly and monthly reports for your consumer surveys and brand surveys.”

A major benefit of the approach, according to Jha, is being able to enhance what is typically a monthly feed of highly relevant, targeted survey data from a tracking study with data sources that are changing on a minute-by-minute basis: “Because these analytics can be run more frequently, they become accurate. Especially if you incorporate some machine learning into the model, because it now contains a feedback loop.”

If you’re an ESOMAR member you can read the full article in MyESOMAR in the digital copy of Research World. If you are not a member of ESOMAR you can join and receive a free copy of Research World 6 times a year or alternatively you can sign up for a subscription of the magazine in our publications store.