Jo Bowman
Published in Research World January/February 2011.
Text analytics is unearthing some gems for businesses – everything from marketing insights to technical breakthroughs – but not without controversy. Analytics strategist Seth Grimes tells Jo Bowman the lines between enterprise and intrusion are yet to be made clear
Since early man first put hand to cave wall, people have been searching for meaning in the written expression of others. Things have clearly moved on since then, but never faster than in the past couple of years, when the value of call centre transcripts and letters from customers has been hugely overshadowed by the huge volume of unsolicited discussion about brands, products and services taking place online. Businesses can see that by listening in on these cyber-conversations, and making sense of what’s said, they can glean much that could give them a competitive advantage.
Seth Grimes is something of a guru on text analytics. Head of Washington DC-based Alta Plana Corporation, he’s also founding chairman of the Text Analytics Summit and widely respected as a leader in this field. Ask him where text analytics fits within the broader definition of market research or business intelligence and he says it’s all about creating insights into customer proclivities, current and potential. Text sources – not just online chatter but also customer emails, survey responses and contact centre notes –can all provide that insight. “If textual content can help you reach research or business goals, you need text analytics,” he says.
In the current business environment, the term ‘text analytics’ has come to mean much more than counting positive and negative reviews, or having someone think about the suggestions that customers make. “Text analytics is a set of software technologies and practices that turn text-sourced information into data for analysis,” Grimes says. “So, text analytics takes market research and business intelligence in new directions by providing the ability to tap text-rich sources, especially online sources.”
Market researchers have been analysing text for as long as anyone can remember, in qualitative research and by poring over news reports, corporate documents and other sources of information. The voice of the consumer has been heard before, but the process of listening, and then shaking-out meaning from multiple conversations, has been a long and costly one. It’s also been one affected by the potential for interviewer bias and interviewees being selectively frank about their habits and preferences in surveys. Online, however, the chatter is largely unprompted, and listening can be done without intrusion.
Online grapevine
Automation means these voices can be heard not by asking questions but by listening to conversations, and it can be done in real time and on a massive scale. An example: “Say the Microsoft Windows Phone 7 product manager needs competitive intelligence and asks, ‘How many people ‘tweeted’ about iPhones this week, what features do they most like and dislike, and how’s overall sentiment trending? Who’s influencing the social conversation and what are those individuals saying about our products? How have they affected Web site visits?’ This information leads to insights related to product design and pricing, messaging, design of campaigns, and so on. But you can’t answer complex, pressing needs like this one with a survey or by sampling social postings. You need exhaustive, automated methods that include text and sentiment analysis.”
The power of text analysis isn’t limited to market research, though. Modern military intelligence and counter-terrorism rely on the ability to process, fast, multi-lingual data and go beyond keyword-spotting to recognise patterns. Grimes believes it’s only a matter of time before a Nobel prize for medicine or physiology results from text analysis of scientific and clinical literature. “The goal is to tease out hidden patterns and correlations, whether they involve protein interactions or help you project.”
Has text analytics transformed a brand? “I can’t say, but it’s definitely essential as marketing and branding efforts turn digital, as efforts move from impersonal broadcast to personal engagement” Grimes says. For these efforts, text analytics and related search technologies form the foundation for an essentially new business – brand and reputation monitoring and management. Text analytics underpins any and every decent listening platform. Companies wouldn’t know what their customers – and prospects and competitors’ customers – were saying online without text analytics.” Professional services firm EDS is said to have reduced the time taken to analyse each of its employee surveys from five person days to half a day through automated analysis of responses, with undiminished quality.
Hot stuff
Text analytics isn’t new, but it is, all of a sudden, getting a lot of attention. The availability of new tools and techniques – and growing affordability – is part of the reason, but the real reason that this has grown into such a hot area is the explosion in usage of social networking. Along with messaging and email, social and other online media provide an unprecedented window on people’s thoughts and habits, and are proving a hugely valuable source of business intelligence.
Where this gets controversial, however, is in assessing when online conversations are public and when they should be considered private. Web scraping, is big business, with data brokers adding discussion boards and social networking sites to the places that used to be searchable in the pre-web 2.0 days – places like the electoral roll, land titles offices and newspaper archives. It’s now possible to match people’s real names to the pseudonyms they use to comment online.
There are, therefore, some big questions being asked in the research community – and the broader web-using community. Should web users be told that their words may be analysed? Should they be asked permission? Where does analysis end and scraping begin? Is anything published on the internet fair game for profit-making purposes? Who owns a comment once it’s made on a web site?
Even the terminology is loaded. One man’s research, mining or analysis is another’s scraping. Grimes says such questions are essential, though not necessarily answerable. “My personal attitude is that if information is online or otherwise available, not coerced or collected via deceptive means and not subject to usage restrictions, then it’s fair game. The vast majority of consumers, who rarely read or reject terms of service, effectively, implicitly agree, but then (publicised) misuses of data have not yet been overly egregious,” he says.
Line in the sand
“There is a line between acceptable and unacceptable, but because public awareness and regulation typically lag technical development, often you don’t discover where the line lies until it has been crossed. When the line is crossed, there’s outcry, a retreat, and then most often the public is desensitised and the vendors (new would-be transgressors) try again.” Grimes points out that he’s an observer of this phenomenon, not an advocate.
This kind of testing what the market will allow in terms of online privacy has been seen before, he notes. In the late 1990s, Doubleclick was using tracking cookies matched to personal information. More recently, controversial US-based company Rapleaf, which links personal information and consumer history with people’s real names and email addresses, has hired a chief privacy officer.
“Try it, step back if necessary, apologies all around, then remix the tech and try again, “ Grimes suggests. “This is how the game is played. Call it a free-market model. How the game could or should otherwise be played, I can’t really say.”
The picture is about to become even more complex, given the proliferation of surveillance cameras, collecting and archiving information that will be usable for many purposes other than that supposedly intended. “We’re not far away, from widespread use of identity-resolution software that will pick up your trail wherever you go, online and in the physical world, and model your individual work, family, and social interactions,” says Grimes.
“Text analytics will be part of this world of extreme data collection, fusion, and analysis, Total Information Awareness Redux. Let’s be aware, let’s act ethically, and let’s keep debating.”
Seth Grimes is owner and principal consultant at Alta Plana Corporation
