The Technology Behind Mobile Research: Browsers vs. Apps

Tim Macer

There’s a fault line running through mobile research technology that makes it not just a disruptive technology but one that is fragmenting research approaches. Forget mobile technology – we need to think ‘mobile technologies’.

At the many mobile research events, pioneering mobile researchers and technology providers jostle over whether the mobile app or the browser is best. According to the Confirmit Market Research Software Survey I am working on, there is no overall winner. Asked which mobile technology was considered ‘most viable’, 39% backed the mobile browser, against 18% favouring apps and 34% sitting on the fence, considering both technologies equally viable.

I am with the fence-sitters. Research just got a little more difficult, because there is unlikely ever to be a one-size-fits-all approach with mobile research in the way there was for telephone or online.

“Researchers need to think about who they are going to reach and why,” says Lumi Mobile’s Andy Lees. ”Is it to gather new data? Or to make more representative data collection in developed markets? Or to make it more convenient for people in developed markets? Or is it to reach people in emerging markets they cannot reach on a PC? In developed markets, are you trying to reach ‘accidentally mobile people’ or are you deliberately targeting mobile respondents?”

A rich experience
For a richer respondent experience, apps have many advantages over the browser-based approach; they exploit all of the capabilities of the mobile device and do so in a tightly integrated way that offers a seamless experience to the participant. Using on-device alerts makes them especially appealing for diary surveys, and allowing participant to tailor those alerts or turn them off for a period makes them a lot more respondent-friendly too. Taking pictures or scanning barcodes is also much easier to integrate in an app.

“Until recently there were two big barriers [to using a survey app] – the cost, and the sample, which was not there” Lees says. “We started providing custom apps, which could cost tens of thousands per app. That does not fit well with most research budgets. But now, by developing a platform for researchers to design their own survey, we have been able to bring the cost down to nearer $1.50 per complete. And now panel companies are deploying apps” so access to respondents is much better now.

He justifies the app only approach that Lumi Mobile is following on quality grounds – it offers a better survey experience for respondents. “Panellists are always very keen to download the app,” he reports. “We know some panel companies have had to limit offering it, because there isn’t enough mobile content for them to deliver”. Unlike other forms of research, it seems respondents are more open to mobile surveys than researchers buyers are – something research companies would do well to exploit.

Location
Apps can also supplement survey response with geo-locational data, either to record where a respondent actually is or to create “geo-fences” that will trigger a survey when a panel member approaches a particular location – a capability beyond the scope of any browser survey.

“Geo-triggering is not that technically difficult,” Lees notes, “but I don’t think the use case has been properly worked out – when you come down to the practicalities of incidence, and how many people do you need to have the volume of triggered surveys, and how accurate it needs to be, in order to be meaningful. It comes back to what the research objectives are.”

Browsers, on the other hand, can offer a universal experience to any research cohort, not just on their mobile browsers but on their desktop browsers too, (though it may be a modified or reduced experience for mobile participants). In developed economies, most mobile phones have a web browser of some sort. The challenge to the technology provider is to make the experience as consistent as possible.

Jason Cazes of Kinesis Survey Technologies says: “Our philosophy is to make sure that whatever device the respondent is using – desktop, tablet, smartphone or feature-phone – they will have a good experience. Respondent experience is critical to reducing bias, whether from drop-outs, response bias or device bias, (where a specific demographic of people tend to use one kind of device).”

Screen size
It is not a trivial matter to deliver a web browser survey tool that behaves gracefully over a wide range of screen sizes and takes into account the eccentricities of individual browsers, but it is considerably easier than having to develop in parallel on entirely different platforms. Very few app developers support more than two or three platforms, and some settle for one.

Jason illustrates the dilemma. “Just when we thought Microsoft was dead, they seem to be making a surge and are gaining market share. You are never going to know what ecosystems are going to exist from year to year. It becomes extremely costly, and the code-porting tools that some developers use (to reduce cost) do not deliver an optimised experience for the respondent on all the devices.”

Browser proponents are pinning their hopes on HTML 5, which should allow them to develop surveys that are as rich and interactive as those currently only achievable in the app. Cazes concedes that tight integration with on-device features is not feasible using HTML, but “HTML 5 is right around the corner. It’s currently quite limited, but it’s going to be there.” In other words, it isn’t yet.

“It all comes down to what you want to achieve, ” echoes Steve August, CEO of Revelation, an online qualitative developer. Revelation recently added mobile support to its online engagement platform but chose to develop a multi-platform app for mobile users. “Mobile in the qualitative space is really about capturing the story and in-the-moment understanding of that story – and the first piece is to capture the story. That means having something that is efficient, engaging and recognises you have a pretty limited amount of finger time with mobile. So you have to consider carefully what piece you are after.”

Steve talks less about respondent engagement and more about using mobile and its brevity to inspire participants. “Some people have taken the bulletin board model but this was built in 1994, and is showing its age. We looked at other social media spaces like Pinterest and found people were truly inspired and willing to contribute to them. The beauty of that visual approach, even if the ‘visual’ is text, is inspiring to participants.”

Developing markets
In emerging economies, there appears to be a misconception that, because many countries have gone straight to mobile ahead of fixed line telephony, mobile self-completion research must be the answer. Andi Friedman, who runs Mobenzi, a mobile research technology firm in South Africa, says: “In emerging markets, face-to-face research is still the gold standard for most survey research. In the parts of the world we work, there is very little applicability of the methods used in the developed world.”

Instead, Mobenzi has focused on harnessing mobile technology to support face-to-face. Andi’s understanding of the challenges his researchers face in the field explains his scepticism about mobile self-completion having much place in developing markets.

“Diversity is crucial in the technology you are supporting,” he insists. “So supporting all the different platforms and rendering technologies that still exist. There are a lot of devices, and when you go global it is even higher.”

Another key factor is building in offline capabilities. “There is a lot of downtime with networks here – sometimes for hours at a time. At that point you start to struggle with the amount of data being held on the handset.” This calls for using data compression to maximise storage, optimising how much data or how many surveys are pushed to any device, and building very resilient retry mechanisms into the data syncing, as one complete might take multiple attempts over many hours to send.

“Battery life is another consideration, when you may be away from a power supply for several days. So using a simpler device with a longer standby time will help, as will using solar chargers or using the charging stations that are starting to appear, which you pay to use. And this means making sure you are not using something with a connector that nobody else uses.”

For Andi Friedman, the benefits far outweigh the challenges. “In general terms, mobile has worked well because the GSM coverage is there, and it is amazing that we do have coverage in areas that you would not expect to find it. You have to be very agile to do research in emerging markets. People are spread far and wide across difficult terrain. Mobile offers the kind of flexibility to respond.”

App vs. Browser: the pros and cons
Key benefits of the mobile survey app

• Engagement. A better respondent experience with better use of screen real-estate and controls can increase engagement and improve response quality
• Integration. Can integrate seamlessly with on-device features such as alerts, barcode scanning and capturing video, audio and photos which are not accessible to the browser.
• Geo-located. Easier to inegrate geo-tagging and even geo-triggering within the survey.
• Advanced surveys. Apps can support more tailored research designs, such as diaries, games, choice models or unstructured interviews. In addition, in conventional online surveys, such techniques, often rely on Flash, which many mobile devices do not support.
• Offline collection. An app can continue to deliver questions and store responses when there is an intermittent internet connection.

Key benefits of the mobile browser survey

• Instant access. Does not require a level of trust and engagement for the respondent to download and install an app before participating, which can restrict apps to work only with mobile panels.
• Coverage. By supporting a broad spectrum of devices, especially simpler devices, it offers better population coverage,and avoids ‘device bias’.
• Cost. It can be very costly to develop and publish a mobile app for a single survey, whereas mobile browser surveys are generally built-in generic survey-creation tools. However, some generic app-creation platforms are now starting to appear.
• Mixed mode. Dual-mode online/mobile survey tools cater fully to unintended mobile respondents in conventional online studies who want to take an online survey on their mobile device.

Tim Macer is managing director of meaning ltd and visiting research fellow at the University of Winchester.