Seven questions for smarter decisions

The authors of Drinking from the Fire Hose of Data talk to John Kearon about how not to drown in information.

I’m sitting here with Paul Magnone and Christopher Frank and would like to talk about the whole subject of data and how it can be used to make smarter decisions for organisations, using your own seven questions from your book. Because that’s your first question: What is the essential business question to be answered here?
Paul: Quite often I think we get caught up in the minutia, in the details, and don’t take a step back and say ‘what problem are we trying to solve?’ So the essential question is better stated ‘I wish I knew . . .’  People typically acquire data, dive into the data, create more data from it, and suddenly you’re under a mountain of data. But they haven’t asked ‘what problem am I trying to solve?’

So businesses, organisations, should always start there. Whatever data is being used for, think ‘I wish we knew x.’
Paul: If, as a market researcher, you’re producing content, and not in line with the business unit, then it’s just more noise.

Chris: I once worked with a general manager who wanted to assess a programme. He said, ‘I need information from the market, I want to understand what the customer thinks of this programme.’ I said, ‘if I come back and they tell you they do not engage with the programme, 0% satisfaction, are you willing to shut it down?’ And he looked at me, straight-faced, and said, ‘you clearly asked the wrong people.’  You have to love this aspect of market intelligence: if the results are right, they’re brilliant, the programme is perfect, the decision was spot on. If the results are wrong, you had the wrong design, you talked to the wrong people and you asked the wrong questions.

So let’s take the second question: What is your customer’s North Star? In the context of the data industry, what is our North Star?
Chris: I give an exercise to my team: in five words or less, tell me what the customer wants. And if we can’t get a consistent answer, we’re not clear on where the customer wants to go, or what their pain point is.

So in five words or less, what is it that clients of market research want?
Chris: I want the person sitting across from me to be smarter than me. I want them to bring a perspective, a different level of experience. That’s how we’re going to get to insights.

Paul: My worst experiences are when I get a big, fat deck as the answer to a question about a market opportunity, a sea of numbers without any perspective. My best experiences are when the market researcher comes in with a point of view, and then has an iterative experience with us in the product prep and the product launch. So we need to illuminate the unexpected as we go and look for the customer’s North Star. It’s much better to do that if you’ve got both people with flashlights and eyes open.

Chris: I’ll even give you four words: I’m looking for a fact-based research partner, someone who’ll stretch my thinking based on new facts and information that other people do not have, or have not consumed in that way.

Paul: The other thing is that there’s an element where you are the steward or curator of the customer voice. There are techniques that you can use to protect the customer’s interest, something that we refer to as the customer impact assessment.

Chris: We came up with four questions: One, how are you going to manage this change for the customer? Two, is it right for the customer? Is this aligned with their pain point, their needs or their aspirations? And then, which one does it map to? Finally, how are you going to service and communicate this? You have to make sure that you’re making the right decision, not only for the business, but for the customer.

Your third question is: Should you believe the squiggly line? Data gathering used to be, in the metaphor of your book, like lowering a bucket into the well and carefully pulling up the bucket of your data, tipping it into the little pre-prepared cups, segmenting your data. And now you’re trying to cope with a fire hose of data coming at you, much of which varies and moves up and down in tiny squiggly lines. How do we cope with this?
Paul: There’s a difference between what’s happening in the moment and your long-term business strategy. And in that moment when that line tacks, much like a sailboat changes course briefly, what is the overall effect of that? It’s putting things in context, triangulating by absolute position changes over time, rather than celebrating the short-term number.

The classic example is political elections. The daily polls which are read as having deep significance. And of course they don’t. A parallel conversation goes on in the financial market between day traders, people who are trying to take advantage of tiny movements in the markets, versus the likes of Warren Buffett, who takes a massively long-term view of investments and has done rather well as a result of it.
Chris: Someone like Warren Buffett may have a tough year, especially if a big part of his portfolio is with insurance, and they have a tough year due to natural events. But over time that overall portfolio is performing well. He doesn’t say, ‘Oh, I should get out of the insurance business this year.’

Paul: Right, but there’s nothing wrong with the day-trader approach. They’re not looking long term. The problem is when you mix the two. Up and down is fine if you’re in the arbitrage business. Up and down is irrelevant if you’re in the long-term business. I think it’s important not to ignore the squiggly line, because business is all about course corrections, and even minor course corrections are relevant. But getting caught up in that is losing track of what’s important. The best metaphor is sailing. You’re making constant course corrections, and yet what’s important is whether you’re heading due North, if that’s your goal.

The fourth question: What surprised you? What has surprised you, and what continues to surprise you, in the world of market research?
Chris: Several things surprise me. One, people still do not put data in context. Two, data is the main character and it’s not the supporting character. We all receive the hundred-page decks and people joke around: wake me when the data is over. I’m looking for two or three surprising findings that they did not expect.

You told a great story the other day about data from a customer satisfaction programme. It just looked really odd.
Chris: 47 different districts in the U.S., and 41 of them looked as we expected. But there were six different districts, scattered throughout the country that looked very different, so I set them aside. And the G.M., to his credit, at the end said, is there anything that you didn’t understand, that didn’t sit right with you? I told him about these six and he leaned over to his chief of staff and said, ‘could you provide me with a list of where we’ve run that pilot?’ And one for one it matched up to these six districts, these outliers that I had set aside. He’d purposely tested them around the country, because he didn’t want to have any regional bias. That ‘tell me what surprised you, tell me what you didn’t understand’ led to validating that pilot programme, which has now become a standard support offering at the company. If a market research firm made that part of their offering – a ‘what surprised you?’ session – it could be a very juicy session.

It’s great when better questions come out of earlier questions. Your fifth question starts to touch on that: What does the lighthouse reveal?
Chris: Effective analysis narrows things down. It says, of the 65 things you care about, these are the five that really matter. You then layer in your business acumen, your intuition and your experience, and focus on those five items. That’s similar to the role of a lighthouse for the captain of a ship. A lighthouse highlights the five obstacles you may need to navigate around. Then again, what are the tides, what are the winds, what is the local weather?

Paul: And once that path through the waters is known, you need to create an example from it: here’s a direction, here’s a route to navigate. Get your teams to understand what they need to do repeatedly to get past the rocks.

Your sixth question is: Who are the swing voters? It has been fashionable in the last few years to focus on lead consumers. You think it’s those that are really in the middle who decide, as it were, the election.
Chris: We could spend another hour just on this. Very quickly: if you want to grow a business, if you want to drive a key metric – embrace it. It’s also very practical. There are clear steps towards identifying swing voters, the ‘neutrals.’ Say they lean positive. Imagine if one or two minor factors shifted enough of them into repeat customers who recommend. Imagine what that would do for your overall sales.

And your final question is three questions in one. You cheated, really, and I think that’s in the best tradition. ‘What?’; ‘So what?’; and ‘Now what?’
Paul: There’s a great quote by Winston Churchill: “This is not the end, it’s not even the beginning of the end, but it is, perhaps, the end of the beginning.” So you’ve found the data that matters, and you might ask yourself what that data really means. But have you created an action plan based on that meaning? That is the ‘now what?’ You have to ruthlessly cut through the data, focus on the business issues, but then say ‘what is actionable from this?’ What is meaningful? Now what? Now let’s dive back into the data a different way. Now what? Now let’s launch this initiative.

Chris: I’ll just end with a quote: Research is to see what everyone else has seen, and to think what no-one else has thought.

Christopher Frank is vice president of business-to-business and communications research at American Express, and Paul Magnone, vice president for business development and alliances at Openet. Christopher and Paul are co-authors of Drinking from the Fire Hose: Making Smarter Decisions without Drowning in Information. John Kearon is chief juicer at BrainJuicer.

Listen to the full audiocast here.