Steve August
Edward Appleton, in a recent article posted on RW Connect, lead with the a title that asked, “Can Online Qualitative Research Be Potentially Misleading?”
Appleton was inspired by an excellent presentation by Peter Totman of Jigsaw Research that showed how online persona were often very different to real-life observed identities, and that people were observed to express differently and sometimes hold different opinions depending on whether they were participating online or in person.
In his RW Connect post, Appleton postulated that there were serious practical implications for online qualitative based on the overall insight of talk that what you see online is only one version of the truth. He goes on to bring up several points, one of which was “If you wish for depth of insight, go offline.”
As someone who has been immersed in in-depth online qualitative research methods since 2004, and who also attended Totman’s presentation, I came away with a different take than Appleton.
While it is generally accepted that people express differently online versus in person, there is more nuance than simply online and offline. Someone in a public facing online interaction – Facebook, Twitter, forums, etc, will behave differently than in a more private online setting. Much as people may express themselves differently in a group setting with strangers than they would when in a one-on-one interaction.
In the last 10 years, I have seen repeatedly with the right study design, moderation and platform it is possible to get closer to the moment of consumer emotions and behaviours, as well as honest depth of expression. Many of the reasons listed above that argue against online can actually be made for using online.
It is true that you lose the visual and body language cues with online and mobile, but you gain access to sustained experiences and expression from participants. This provides great opportunity to go deeper in understanding people’s behaviours, experiences and emotions. While every research medium has strengths and challenges, absence of depth is not necessarily a default challenge of online research.
Many sensitive topics are actually better suited for online qualitative, as people have a layer of anonymity that enables to express opinions more fully. Using web and especially mobile, participants can engage much closer to the point of experience, before they have processed and rationalised. Context can be brought into the mix through a variety of activities that help enable participants to frame their experiences.
With interactions and activities that are focused on understanding people around a certain topic, and where one-on-one interaction are part of the studies, we see less contrivance.
The way online qualitative research activities are designed and sequenced can also serve to assist in seeing the authentic person. In online qualitative, researchers have the opportunity to control the environment – utilising both one on one interactions and group interactions to serve as a check and balance for how a participant expresses.
Ultimately, any type of research, online or in person, can potentially be misleading. Whether it is a leading question, a group dynamic gone awry, non-representative social media feeds, or a poorly designed survey, a main part of our value as researchers is to understand and control the dynamics of our methods and mediums.
Read Edward Appleton’s article Can Online Qualitative Research be Potentially Misleading?
Steve August is CEO at Revelation
2 comments
This is such an interesting discussion. From my perspective as an researcher doing both online and f2f qual, I agree that multi-modal is the gold standard.
As for the kind of depth to expect from online qual, it comes down the kind of online qual research that you are talking about.
A lot of the promotion of online qual has focussed on its superficiality – the idea that you can achieve insights quickly and without attempting to engage with people gets far too much airtime in my view. If that is the kind of online qual the original paper referred to then yes.
However if you design it right, in my experience, online qual research CAN give you more depth than face to face for 2 reasons:
1. People can read the moderator’s question, and then go away to think about their answer for several hours before coming back to the discussion and posting. You get a much more considered response than the ‘answer me now’ hothouse environment of a face to face group.
2. Secondly, people can express themselves online without being interrupted. We find that written posts are typically longer and more detailed than the discourse of our f2f groups.
These arguments to me are the main reasons to do online qual – the ‘geographical reach’ benefit comes in third.
Perhaps there was more to the ‘multiple selves’ argument than I have understood through these posts. Yes, people present one ‘self’ online; but they present one ‘self’ face to face too. We all do it. Perhaps we researchers know more about WHY people choose to present a particular ‘self’ in a f2f group (group bonding being the simple answer) than we know why people choose the ‘self ‘ that they present online?
Having said that, I do have a linguistics background and am perhaps ‘linguistically attuned’ to quote Edward. Perhaps we as an industry need to develop and share some of these linguistic skills?
Steve – thoughtful response, thanks. Sure agree that folk are more likely to be less inhibited in private forums than open ones such as Facebook.
Subject matter to me is another variable. The areas Peter Totman focussed on were emotive – racism, immigration I think – areas where two things likely come into play: you may feel strongly, but not feel comfortable that your view will be approved of by others. So it makes sense that you say different things in different circumstances – both are true of how people in that situation may feel, truth is plural, potentially contradictory.
Additional factor to throw in – the oral versus the written, whether the oral is a more uninhibited form (fleeting, unrecorded) per se than what is documented in writing or visualised digitally. Digital documentation is, as we know, seemingly for ever, and who knows what the word privacy is worth.
To me, accessing a full picture (“the truth”) means – as you say – observing and listening over time, taking in attitudes and behaviour, so (no surprises) effectively triangulating. Context is key – seeing how our views shift according to where we are, who we are with, how we feel.
Another check is asking other people’s opinions about what a person has said – looking for surprises, contradictions maybe.
I guess using online qual. is great for the known advantages – geographical reach, mobile= in-the-moment, longitudinal possibilities, cost – but I would still suggest that if a project has involved NO face-to-face, quasi-ethnographic component, the danger of superficiality can be real and present, dependent on the objective.
Curious as to others’ views.