Arun Joshi, Sagar Tamang and Himanshu Vashishtha
First published in Research World January 2009
Acquiescence bias, or the tendency for respondents to agree with whichever answer is presented to them, is a big issue in the Middle East which is why we have developed this new technique
Measurement error is always a hot topic for researchers and two factors that most strongly influence the quality of responses are acquiescence bias and social desirability bias. Acquiescence bias is a systematic bias stemming from the tendency of some respondents to agree with whatever is presented to them. This can result in a strong skew in response towards the upper end of a scale which means the scale loses its sensitivity when it is used for, say, comparing one item with another.
Acquiescence bias is most visible in surveys amongst Arab respondents in the Middle East. In fact, Saudi Arabia is one of the countries showing the most severe case of acquiescence bias worldwide along with Brazil, India and South Africa. At the other end of the spectrum are countries such as Germany, UK, Japan and South Korea where this bias is of little concern.
We address here the issue of acquiescence bias specifically in the context of Arab respondents in the Middle East but findings and recommendations could well be of universal value.
Underlying reasons
One critical factor for acquiescence bias within a society is its level of collectivity. In cultures that are marked by a high degree of collectivism, individuals feel it as important to consider what people might think and say about their actions, and this creates a bias.
Another factor is politeness, the level and nature of hospitality displayed in a society can have an impact on acquiescence bias. If a face-to-face interview takes place at the respondent’s place of residence, the respondent will treat the interviewer as a guest and exhibit behaviour that befits a good host. In a culture that preaches being positive, ‘negativity avoidance’ tends to create acquiescence bias meaning that whilst a respondent might well have a negative feeling or reaction, they would tend to avoid giving a negative response.
Other factors such as economic prosperity, consumer confidence and a high degree of materialistic indulgence tend to add to acquiescence bias.
Some solutions
Different approaches have been applied to try to reduce acquiescence bias. Examples include using a reverse order of points on scales, or negatively worded instead of positively worded statements. Other approaches involve using a positively imbalanced scale, expanding the number of points on the scale, rewording the question so it not leading, and incorporating projective techniques.
Generally they have had limited benefit because they all involve using a verbal or numerical scale which involves a gradation of points from a positive to a negative end. Respondents when faced with such a scale have the obvious option of reflecting courtesy through choosing an answer that is a couple of notches higher than their true response.
We reviewed various qualitative research projects looking for ideas on how to go forward. There was consistency in how researchers can gauge realistic feelings though applying projective techniques. Enabling techniques which relied on visual stimuli seemed to work more effectively than techniques involving verbal means.
We therefore aimed at developing a scale that uses pictorial points and projection instead of verbal or numerical labels. Furthermore, points on the scale should not have any obvious order from ‘positive’ to ‘negative’ or from ‘high’ to ‘low’ as perceived by the respondent. Additionally, there should be a way to convert the nominal scale subsequently into an equivalent interval scale to facilitate data analysis.
The new scale
We set about constructing the Joshi-Tamang-Vashishtha Scale around a set of multiple pictures. We propose to illustrate the method by using a survey involving assessment of a product concept with housewives as the target respondent as an example.
The seven steps involved in interviewing the respondent when shown the product concept are as follows:
- The respondent is shown the product concept.
- She is shown a pre-selected set of pictures, each depicting a particular type of women.
- She is asked to provide a response relating to ‘intention-to-buy’ and asked which of these types of women is most likely to buy the product.
- She is then asked in the same way questions about other parameters of evaluation, such as likeability, relevance and uniqueness.
- After all the questions have been posed, the respondent is asked to say which of the various types of women is the most and least similar to her. She is then asked to place each of the other pictures in a ranked order reflecting how similar each of them is to her. A specific method, involving assigning a number of points between 100 and 1 to each picture, is then used to determine inter-point distances between the pictures.
- The respondent’s answer regarding ‘intention-to-buy’, in the form of the picture that she chose in step 3, is then converted to a numerical response based on the number of points she assigned to it in step 5. Similarly, the respondent’s answers to each of the other evaluation parameters are translated from the picture chosen in step 4 to the corresponding number of points assigned to it in step 5.
- The numerical response can then be converted to a scale of any number of points as desired through rescaling.
Note that this conversion into an interval scale – in terms of how the pictures are sequenced, as well as in terms of the points that are assigned to the notches on the scale – is unique to each respondent.
Developing the scale for the Arab world
A key task in creating our scale for the Arab world was identifying specific pictures that could be used as the different points on the scale. We decided to work on developing a scale that can be applied for a housewife as the target respondent for this. Here is what we did.
- We gathered numerous pictures showing a wide range of different types of Arab women.
- We then classified the pictures into several groups based on the type of women portrayed. This was done through small-scale primary research involving depth interviews. Ambiguous pictures were removed and any overlaps between groups were also discarded. This resulted in a total of 12 groups of pictures.
- We examined each group of pictures intending to select one that most appropriately represented the particular group. After a series of iterations, it was concluded that there should be a set of four pictures for each group to properly reflect the essence of the group.
- We needed to identify which of the 12 sets could be chosen finally to represent points on our scale and this was done through a quantitative survey involving the same method of assigning a number of points as described earlier. In conclusion, we identified five sets of pictures to be regarded as points on the scale.
- Through another large-scale quantitative survey, we re-established that the five sets of pictures were indeed an appropriate choice as the points on the scale.
The development of our scale for the Arab world was followed by an exercise to validate its effectiveness in reducing acquiescence bias. Results revealed that responses using our scale were well spread out compared to responses from using the conventional verbal scale. The validation exercise reaffirmed that the Joshi-Tamang-Vashishtha Scale can prevent the clustering of responses towards the upper end of the scale by considerably removing acquiescence bias.
The approach adopted by the Joshi-Tamang-Vashishtha Scale deals with acquiescence bias by ensuring that the respondent is not able to express any such bias. This scale can easily be used for any research application that needs to measure opinions, attitudes and other hypothetical data.
With appropriate localisation of pictorial points, the scale can also be effective in any part of the world and it can be adapted for any consumer segment as well as housewives.
The Joshi-Tamang-Vashishtha Scale is dependent on factors such as lifestyle and social customs, and so it may be necessary to keep updating it as and when appropriate.
Arun Joshi is regional head of knowledge management and client development at The Nielsen Company, Eastern Europe, Middle East & Africa. Sagar Tamang is business unit head and Himanshu Vashishtha is managing director at The Nielsen Company, UAE.
