Measuring Minds

By Kevin Gray and Gordon C. Bruner II

Marketing scientist Kevin Gray asks Dr. Gordon Bruner, a noted consumer psychometrician, for advice on how to measure consumer attitudes.

KG: Marketing research has seen many changes in the past few years. In your opinion, which have been most consequential?

GB: There are several that come to mind right away.  First, the internet makes gathering data via surveys or social media much easier.  Second, the ability to apply powerful statistics and models to analyze data is also easier now.  A third innovation has to do with neurophysiological methods, although I believe their many limitations will lower their widespread value to marketers for years to come.

KG: Some marketing researchers feel that traditional surveys don’t actually work because consumer behavior originates in the subconscious. They feel we need ‘implicit’ measurements of some kind to really understand why consumers react and behave the way they do.  How would you respond to these views?

GB: I give very little credence to the notions of the subconscious or to any ‘theory’ that says people don’t know why they do what they do or, at least, they cannot express it. Humans would not have survived as a species if there was not reasoning that helped us make better decisions along the way. People are rational in the sense that there are reasons for behavior even if those reasons seem irrational to others, even if emotion is part of our reasons, and even if tiredness or other situational factors impair our reasoning process. One of the purposes of market research is to figure out the reasons people do what they do. If there is no reason to what people do then there is no point researching them.

KG: Some also feel survey research is losing its value in the age of social media, extensive transaction records and other ‘big data’.  They feel it’s no longer needed.  What are your thoughts about this?

GB: Analyses of social media and use of ‘big data’ have their place but they should not be viewed as replacing good survey research or experimental methods because they do not provide accurate information about many of things that good primary research does. For example, I view data that comes from social media as qualitative and preliminary. Analysis of that data can provide initial insights into trends and other issues but it would be incredibly naïve to stop there and base decisions on it except for minor issues.  Instead, insights based on social media research should be followed up with the scientific confidence and accuracy that come with good quantitative research. As for ‘big data,’ it can help understand some of the WHAT and maybe even the HOW of customer behaviors. But, if the marketer wants to understand WHY people do what they do then they need to get into the mind of the customer.

KG: You mentioned ‘getting into the mind of the customer.’  How do you suggest doing that?

GB: Primarily, I am referring to the use of psychometric techniques. They play a unique and valuable role that cannot be replaced with social media research or ‘big data.’  In case the term psychometrics is not familiar, it has to do with the science of measuring (metrics) psychological constructs (psycho). By the way, to be clear, it has absolutely nothing to do with Freudian psychology, psychoanalysis, hypnosis, projective techniques, or other long-discredited forms of understanding what people think, feel, and plan to do.

KG: Perhaps you could give us a very brief history of psychometrics and how it has evolved in marketing research over the years?

GB: For many decades in psychology, there has been a realization that no one question or statement (what I call an ‘item’) can perfectly measure psychological constructs in the mind.  Thus, multi-item scales were developed. (It is analogous to the use of triangulation in ground surveying in order to determine the position of something.)  Along the way, methods for assessing the quality of the multi-item scales were developed in order to quantify how well a scale was capturing what it was supposed to measure.

As for when multi-item scales began being used in MR, I don’t know for sure but a reasonable guess is that it was in the early 1970s with William D. Wells and his study of lifestyles. I do know that during that same decade, usage of multi-item scales in the scholarly marketing journals became more prominent.  Then, usage exploded in the 1980s.  Not only were scales created for measuring lifestyles and all sorts of attitudes but also for many other psychological constructs such as motives, values, emotions, intentions, etc.  In those early days, researchers borrowed scales that had been developed by psychologists but over time marketing professors began developing their own measures, especially when it came to constructs that were more related to marketing such as products, prices, promotion, and purchasing.

KG: What are the key things marketing researchers need to do to make this kind of measurement work?

GB: That is a very big topic and one that is difficult to briefly summarize. One of the key mistakes is to assume that any one question adequately and precisely measures a construct of interest. When measuring things in the mind, there are rarely if ever 1-item measures that are very accurate.  In fact, we still don’t have good ways to measure the quality of single-item scales.  At the other extreme, lengthy sets of questions such as used historically in psychology are not necessary either.  High quality scales with just two to five items are commonly developed by marketing scholars now.

Prior to creating new scales, however, I urge the borrowing of good quality scales from past research.  Finding such scales developed in industry is challenging due to proprietary concerns. But, it is just the opposite with the scales used by professors and reported in scholarly journals. There are several thousand multi-item scales measuring a very wide variety of constructs that have been tested by scholars and then vetted by journals when/if the research is accepted for publication. Admittedly, there are situations where multi-item scales are not needed or when their usage is not possible. But, even then, I would ask the question of those designing a questionnaire – what is the quality of the survey questions you are using?  If the designers and others they work with cannot provide a sufficient answer to that question then users of the findings ought to be very cautious about making decisions based upon the data collected.

KG: Are there special challenges with psychographics for marketing researchers who work internationally and cross-culturally?

GB: Absolutely, there are tremendous challenges!  That is another very big topic that is worthy of a separate discussion.  Suffice it to say now that trying to measure the same thing in multiple countries makes sense but to do it correctly is not for the faint of heart.  For example, just because a measure has been validated in one country does not mean it will be a valid measure in other countries. Translating scale items introduces unwanted variance and items might need to be added, rephrased, or deleted.  Luckily, there are high level statistics for helping to assess measures and refine them so that what is measured in one country is the same as what is measured in another.

KG: Beyond understanding and using psychometrics when appropriate, what can practitioners do to improve their research?

GB: I am concerned about how much researchers know about the findings of studies conducted by marketing scholars. I cannot speak about every MR agency, consultant, or department but I do know from what see discussed in social media by people claiming to be in MR that a basic knowledge of marketing and/or customer behaviour is sorely lacking. Part of MR is using good science to gather data but another important part is understanding marketing itself. One of the most complicated parts of understanding marketing is having an adequate knowledge of the people who are being marketed to, particularly if they are the ultimate consumers.

The good news is that there are hundreds if not thousands of professors who are doing research of consumers and generating information that can be useful for a wide set of companies. A lot of this information is free!  Given this, a part of good market research is knowing the scholarly literature that is relevant to the current project. This is secondary research and it can inform, shape, and guide the planning of primary research. Whether it is a staff member in the MR department, a consultant, or a professor, there should be people brought into the research process who have a command of the scholarly literature most relevant to the purpose. MR should benefit from the light that is shed by scholars on topics rather than continuing to recreate the wheel in the dark.

KG: Let’s fast forward to 2030…how do you think survey measurement might be done then?

GB: I hope by then that Item Response Theory and adaptive survey techniques increase the quality of measurement while at the same time reducing respondents’ time. My concern is that if practitioners will not even use current psychographic techniques then they may shy away from methods that require even more skill to understand and use. I urge people in MR to keep up with these developments and use them when possible.

KG: Thank you, Gordon!

Kevin Gray is president of Cannon Gray, a marketing science and analytics consultancy.

Gordon C. Bruner II is Professor Emeritus at Southern Illinois University.  He is also author of the Marketing Scales Handboook.