Preriit Souda
Dear Diary,
Last night I had a big fight with Ulov (a Chinese friend) while discussing economic conditions in India vis-à-vis in China. I picked out a few studies which showed India as a better option. Ulov questioned the study itself and raised concerns over the quality of the study; he asked if the interviews were conducted correctly and with the right people. Another day, a friend of mine (manager at a CPG company and a consumer of market research findings) was concerned about some of the performance indices going down and questioned the reliability of such surveys. Though I was initially defensive, later I questioned myself, “Can we (market researchers) swear by our data?”.
Personally speaking, I do find instances when certain characteristics of a particular section of society, as reflected by data, do not make any sense. While researchers often blame it on interviewers (FYI-more than 90% of interviews conducted in India are face to face), data collectors/interviewers blame it on researchers. Speaking in an Indian context (as that’s where I get to work most) I feel that there are three sides to this problem and all are equally responsible for it; the three sides being the research agency, the client and the data collector.
First let’s take on clients. Clients often come up with vague requirements (say extremely long complex questionnaires) with even laughable amount of funds sanctioned for such studies. Let’s give a recent example of the impact of such acts; a lady in my neighbourhood had once been interviewed for a market research study for 1 hour and 15 minutes and in the end interviewer said “Thank You” and walked off. Another time she was given a 20 rupee soap for a similar study. After these two incidents she has sworn never ever to respond for any market research requests. She also asked her locality officials to prohibit any market research interviewer from entering their complex. Giving a 20 rupee soap or fake smile as a gift to a SEC A (upper class of society) for a questionnaire that took 75 minutes to complete is a ridiculous joke!
This is one of the reasons why SEC A respondents tend to avoid market research requests. If I place myself in the shoes of the respondent, I, too, will never entertain any such request. In fact, often I have seen that people who commission such studies, themselves have no idea of how long it takes for a questionnaire to execute and I wonder if they themselves would be willing to answer for so long and that too for a soap which they won’t prefer in the first place. There have been countless instances of colleagues telling me that respondents hate us (market researchers) but we never pay attention to such issues. I have often wondered as to what’s the point in asking a respondent beyond a point when you know that he/she would no longer be interested or have the energy to answer meaningfully. Why do our studies evoke emotions of hatred in people whom we derive our bread & butter from? Add to this what’s the point in carving out strategies out of such insights?
Now let’s put research agencies in the box. We as agencies are so mad on winning projects that we often become ‘Yes men’ to vague requests of clients. Plus add to it the towering pressure to win these bids that we reduce our quoted prices beyond the logic. In fact often the price at which we do business is quite laughable. Ashok Das (President, Market Research Society India) raised an interesting topic of low research costs in India at Best of India-ESOMAR 2012. Quoting ESOMAR’s global pricing report, he expressed concern about India having one of the cheapest research prices in the world. Honestly speaking, I don’t see any possibility of all agencies uniting together to solve this malice.
There needs to be other ways to differentiate our services and stop our products from becoming a commodity. Coming back to data collection, I feel that often researchers do not stress their brain cells in order to add a bit of ingenuity in devising ways for collecting data. Now given low costs and long questionnaires, data collectors add their spice to the already messed up Indian curry. Add to it, the falling quality of interviewers fuels the problem. I am a bit soft on interviewers as I believe that these are low cost manual laborers who are simply expected to do what’s being told to do; based on what’s being planned by researchers and clients. If the other two devise studies properly and place proper checks in place, 90% of the problem will be solved.
While I may have been a bit harsh in criticising my fellow colleagues and clients, which I in someway regret; I urge passionately to think about these issues with heart and mind at right place. If we don’t think about these issues (and many others like these), 10 years down the line, the MR industry won’t exist. Let’s sweat today for a brighter tomorrow.
Controversially yours,
Preriit Souda
PS – If anyone is hurt by my words, I can apologise personally. Yet I stand by my words as I am passionate about my industry and data quality is always a dear topic to me. Please note that these are my observations. I have not conducted any detailed study to substantiate these observations.
Preriit Souda is a consultant at TNS Global, the views expressed are solely personal and do not reflect views of TNS or associated companies.
3 comments
It’s tough to ever “swear” by data, namely as the data you will always collect is always going to be subjective of the person collecting the data. What I mean by that is that is the results and data collected will not intentionally but always be biased in some way. Whether it’s the kind of questions asked or which data is collected and collated etc
“You get what you pay for” is a cliche that we are all aware of – “garbage in and garbage out ” is another thta has been used to describe what happens with poor quality data captured.
What is paid for and what you get is undermining what is happening in India with respect to the quality of data being collected.
The Industry at large needs to pay interviewers a wage that is matched with the skills and integrity that would provide good quality interview utlisiling Electronic data capture as much as possible across face to face methodologies.
That way we have a better chance of capturing the data and being able to monitior quality both at point of capture and also validating the interviewer and data in an effective manner.
We have to improve the engagement between an interviewer and a respondent, and that means improving both what interviwers get paid and how they are treated, and also how to give a respondent a better experience.
Researchers need to push back on interview length and what is asked as agree with comments that length of an interview affects quality (reduce interview fatigue and gaining a more representative sample.)
This is not unique to India, but then India has a relatively low cost for interviewing that has hindered investment in technology and providing a better interviewer engagement.
Collectively we all must help improve this and be proactive.
It wont happen without the widespread support of Clients and Researchers – The MR Industry needs to be united in upholding the importance of quality of information collected and used to gain the valuable insights required and improving how interviewers are engaged and how they then engage with respondents.
I think the idea of a SEC A client agreeing for an interview and expecting something in return itself is fallible. I dont think that a soap or anything else would be a concern, it’s the amount of time which matters…
Well as far as India is concerned, I believe that barring the conditions that have prevailed during the last 2-3 years, the taxation system(skewed and evasive), the preparation of annual reports(skewed), the rate of procurement of licenses and profits from other departments used to make so much of money that money put into marketing resources was not much of a concern.
Now I will tell you why your researchers, data collectors and interviewers fail.
If you look at the bigger picture then obviously everyone wants everything(quantity/quality) at the cheapest price, which almost no company can offer. It is rare for people to take a perspective of the costing a company might have to incur in order to manufacture a product and factor that in. So the products actually never meet the expectations of people(Either price or quality is compromised, getting both is very difficult). Hence eventually over a period of time it creates an impression of surveys just being a waste of time and not keeping up to people’s expectations, which are infinite and are mentioned in the survey’s too!!!