Research in the Digital Age

Surinder Siama

Sean Bruich, head of Facebook’s Measurement Research group, on its ambitions in the research space.

What types of research is Facebook working on today?
There’s a number of analytic and research teams at Facebook. What my team primarily focuses on is two-fold: the first focus is developing research approaches for measuring and understanding consumer preferences and behaviours. This is particularly salient in the marketing world today, where a lot of companies are trying to figure out how best to use Facebook and other online channels to reach and communicate with their consumers. There’s also a big emphasis on evaluating marketing efforts across multiple channels and for objectives beyond driving traffic online. I think there’s a real opportunity for Facebook to help there. Another focus is looking for ways we can use Facebook to better understand the world around us. With 750 million users across the world, there’s a lot we might be able to learn – what people care most about in a given time or place, how trends get started or peter out, and how friends share and influence one another,

What do you think about when trying to achieve those goals?
There’s a neat opportunity to think of Facebook as a research or information platform. There are a few reasons why we’re excited about it. The first is the global footprint and the sheer number of users who visit the site every day. No matter who you are interested in hearing from, there’s a lot of people available – globally but also locally.

The other really interesting thing is that when people use Facebook, they are their real selves. They connect with their real friends. They are sharing real information about their lives – which puts people in a very honest context. You’re not intercepting people out on the web where they feel anonymous and maybe suspicious about why you are asking them about their use of products or how they feel about the world.

What also makes Facebook interesting is how engaged people are when they’re on the site. The home page is an active experience: users are ‘liking’ and commenting, they’re chatting with their friends. It’s not that much of a jump for somebody to give feedback about how they feel about something. Consequently, participation in research on Facebook is incredibly high, and the response rate on Facebook research polls is orders of magnitude higher than standard pop-up surveys. We actually see incredibly broad participation across lots of different types of users(.

When you look at online research methods today, they are not all that different from the ways research has been done for the past 50-70 years. While it’s easier than ever to build, programme, and field a survey, that doesn’t mean that it is any less burdensome for participants. One of the most important things we’re doing is trying to make participation in research a positive, rather than interruptive, part of the user experience. We find that this actually results in better data, because a much more representative set of people are willing to participate when research is low burden.

How do Facebook’s research polls work?
One approach we’ve taken to making sharing opinions much lighter weight is research polls. We invite a user to answer a quick, one or two question poll right on the Facebook home page. We call these research polls and they are great for getting fast feedback on a variety of topics.  People like sharing their opinions. But I think that it’s pretty obvious that most people are not interested in taking a long survey online. If you think about the user experience for an online survey, what you’re typically asking someone to do is stop what they are doing, go somewhere else, and take a long, boring survey. Making the research fast, easy for the consumer, and in line with what they are doing is really important.

We’ve done a lot of validation work to understand the benefits of this type of approach. For example, we can look at the representativeness of these polls and see that, relative to a long survey, a much broader, diverse set of individuals respond. We can also assess external validity by asking a set of questions on Facebook and comparing those responses to a well-established methodology. Some of the data is tracking public approval polls, for instance of President Obama. What we find is an extremely high agreement between the data we get on Facebook and the public data collected by other polling companies. Our goal is not to become a research company, not replace a Gallup poll or an economic outlook indicator. But we do want to work with others to find ways of helping people understand the world.

Why wouldn’t you want to replace the Gallup poll, etc., if your data is just as good?
We’re not designed to be a research company. We believe really strongly that the way that Facebook is going to succeed is by partnering with other companies. We have this platform perspective that we can’t really solve all the problems in the world – eg, we’ll partner with gaming companies like EA and Zynga rather than build a bunch of games ourselves. We try to take a similar approach with research as, for example, with the recent white paper written with Nielsen on the power of social in advertising.

We don’t see a Facebook poll or any Facebook research product as a monetisation opportunity. We don’t sell user data. We don’t sell our research products. The polling platform is not open to everybody – it’s not a self-service tool. We want to make sure the questions being asked are appropriate and well-designed from a research perspective. We limit how many polls are asked at any one time to very small numbers. We’d consider partnering with research companies – we’ve already done that with Nielsen and we’re working with others, such as Millward Brown and comScore, on a variety of interesting research topics.

Lots of online research companies are moving away from surveys and focusing instead on analysing text from social media sites and blogs.
Definitely, and there are benefits to analysing techniques that take advantage of existing data. Many marketers are eager to build their brands through earned media on social sites, so this is of great interest. A lot of research is focused on counting or categorising the discussion about brands or products. It seems far more important to understand what types of people are seeing those discussions, and what impact it has on them. If you think about text analysis as an understanding of the stories out there about a product or brand, you really want to know how many times and to whom those stories were told.

Additionally, from a research perspective, it’s also important to understand the bigger picture of what you’re measuring. People’s expressing themselves in a blog post or a status update may align with their broader set of feelings or opinions some of the time, but there are also instances when discussions don’t tell the full story.  Hearing people’s opinions through polls or surveys is still going to be important.

What are some research studies you’ve done with partners?
Two come to mind. Nielsen did a very interesting study looking at the impact of social advertising. They found that social ads on Facebook increased the likelihood for consumers to remember the ads by 60%, and doubled the likelihood that a consumer would take away its core message. comScore also just put out an interesting paper looking at the fans of brands on Facebook and the friends of those fans. It turns out that those “friends of fans” are really important – they are between 30-80x the size of the fan base, and they share a lot of the same behavioural and psychographic characteristics of the fans, including an increased willingness – more so than the average Facebook user’s – to consider or purchase that brand’s product or service.

Is there a conflict between user privacy and the demand from research and other sectors to collect and analyse conversations?
I really don’t think there should be a conflict. When you look at the type of stuff we’re going to do on Facebook, it’s going to typically be aggregate, certainly no personally identifiable information. Polling is very much an opt-in opportunity and it ends once the user has answered the polling question. We anonymise that data and users are not re-contacted. And here’s why I don’t think there’s a conflict: if users don’t believe in what the polls are doing, if users are not happy with the research experience, then the whole thing breaks down. Facebook has a heightened obligation because users are not on the site as research participants. One of the reasons we developed the polling tool is because users actually like answering that question. On Facebook, what’s private is private. We never sell user data.

Based on what you have learned so far, what are the best ways for brands to connect with consumers on Facebook?
I think the research world needs to catch up with what’s going on in online in determining the best ways of measuring all of this activity, whether it’s online advertising or earned media. But online also needs to measure itself in a way that integrates it rather than isolates it from all the other activities that marketers are interested in, i.e. television advertising, offline spend, etc..

In terms of what’s most important to advertisers, there are a couple of things. Absolutely, social advertising is of incredible interest to advertisers. Understanding what it is about social, whether ads on Facebook that incorporate social context about your friends’ interests, or the earned media component to see word-of-mouth from friends on Facebook or elsewhere online – what that does to a consumer, what information is conveyed and how it influences the decision-making process, whether it is for the purchase or towards feelings about something else that is going on around the world – that is going to be a really important aspect of research in the coming years.

What have we learned so far about what makes advertising on Facebook powerful? The social aspect seems to be a critical component of making advertising interesting to users, and also making it relevant. Seeing some information about your friends is always going to be more interesting than seeing a piece of information out of context. In an independent study by Nielsen, they found when an ad was social, users were 68% more likely to remember the ad and twice as likely to take away its message. This makes sense – in the offline world, most information coming at you has a lot of context: someone tells you a story about an experience with a product or a brand. That isn’t always the case online with advertising.

So researchers need to catch up – what should they focus on?
It’s a challenge because online advertising measurement doesn’t even have the basic question answered: who are you hitting with your advertising? This is partially because online grew up primarily as a direct response business where you didn’t really care who you were hitting as long as they clicked and bought whatever you were selling. We as an industry haven’t provided the right metrics to help brand advertisers understand how their media is working and, even at the basic level, how many people were hit with an advertising campaign. Cookies are not people. Browsers are not people. And impressions are not people.

One of the interesting things we’ve been doing is working with Nielsen. Their Online Campaign Ratings product is designed to extend the same type of campaign audience measurement they’ve been using for the last 40-50 years on television with reach and frequency among demographic segments. That’s obviously just a first step, but it enables advertisers to understand who in aggregate is seeing the ads and begin to then measure the impact of that advertising.

For Facebook, isn’t the atomic unit for advertising a person?
That’s a really important aspect of Facebook – the authentic identities of everyone on the site. But that isn’t the case for most online advertising research. Many are still counting cookies, browsers, etc. – despite the mounds of evidence that those don’t correspond particularly well to people. But one of the things I have learned over the last few years working in this space is that solving something for one site doesn’t solve the goals of advertisers, which is to understand how they are reaching consumers. So we as an industry have to figure this out more broadly.

It sounds like developing a unified metric that works across both online and offline media will be messy.
When you think about what you’re really trying to understand when you run a media campaign, everything starts with understanding who was exposed, who may be engaged with the ad, and what impact that advertising had on changing attitudes and behaviours. It’s critically important that we start to build equivalences between various forms of media, and that we understand the basics of who and how many are seeing ads across sites in a way that’s standardised across media platforms.

Given Mark Zuckerberg’s recent announcement of a shift from size to engagement metrics, what are the implications for your area of Facebook?
There has been a lot of focus among brands on building fan bases on Facebook: acquiring users interested in your products and connecting with them. That’s really important. We just worked on a paper with comScore. What they found is that, as big as those fan bases are, the real opportunity lies in the friends of fans.

If you think about what Mark said, growth is really important, users are really important, but once you have those users what matters is engagement. So the goals should really be activating those users, helping people to feel more comfortable about sharing, and enabling them to connect with their friends in new ways. I think brands can take a lot of inspiration from that.

You can hear the full audio interview here.

Sean Bruich will speak at ESOMAR’s 3D Digital Dimensions event taking place from 26 to 28 October in Miami.