Sampling – research’s forgotten child
Research provides value to innovation with new ideas, finding unmet customer needs and profitably developing new products. However, sampling traditions reduce how much value research can add to innovation.
Digital platforms, more IT power and design thinking (amongst others) has innovated data collection and analysis. Conversely, sampling has been left behind. Online panels and social media have evolved the sampling process. But how we decide who to include in research has been comparatively stagnant.
Sampling convention – innovation’s BMI
Samples for innovation research usually reflect a specific target customer profile. To be included in such samples, people must reflect the ‘typical’ or ‘average’ target customer’s demographic, behaviours and attitudes.
This approach (first known as ‘a monograph study’) is based on Belgian astronomer Adolphe Quetlet’s ‘average man’ study. This was the first known use of averages to summarise large data sets.
Quetlet’s ‘average man’ study led to him inventing the Body Mass Index (BMI). The BMI calculates if your weight is healthy based on height, weight and age. However, the BMI is viewed as flawed. It ignores nuances like muscle and fats differing weights in favour of simplicity. Resultantly, BMI classes World Heavyweight Boxing champion Anthony Joshua as nearly obese.
The value creation risks
For reasons like the BMI’s poor interpretation of an elite athlete’s health, reliance on target customer sampling limits the value innovation research can create:
- Typical/average people create average thinking. Average thinking = average insight. Average insight = average innovations. Average innovations = failure
- Target customer profiles reflect typical/average customers. However, the amount of target customers with a typical/average profile is low. In 1950 US Air Force Lieutenant Daniels showed the issue this causes.
- Daniels studied why US Air Force planes had 172 safety incidents in one month. During his study, Daniels measured 140 physical dimensions of 4,063 pilots. No pilots had average dimensions on even 10 of the 140 measured. Therefore, the US Air Force’s cockpits were designed vs. a non-existent average. This error cost lives in air travel. It will cost millions in business
- Innovation research typically uses samples based on geography and category behaviour. This seems to make sense. Why do research where you aren’t active and with people you won’t sell to?
Here’s why. The world isn’t defined by geographical borders and product category labels. Far from it. The internet means society is borderless beyond words compared to when British haberdasher John Graunt first used a sample in 1622. Sampling, to its detriment, has ignored this.
The value creating solutions
Target sampling had a diverse birth combining astrology and haberdashery. Its routes to creating more value are equally diverse:
A table tennis rebel
Olympic table tennis player Matthew Syed’s (AMAZING) book Rebel Ideas argues that diverse perspectives broaden how a problem’s viewed. This broader view creates a wider frame of reference for creating solutions. This means a group of individuals with the same views are collectively less intelligent than a group of individuals with diverse views.
This suggests that target samples for innovation research can better solve problems, and create more value, by having more diverse profiles within them. It’s no coincidence that forecasting guru Phillip Tetlock cites being open to diverse information sources as a trait of the best forecasters.
An inclusive console
In the early 2000’s, Nintendo couldn’t compete with Sony and Microsoft. Games industry convention meant that consoles should compete on power. Nintendo’s game designer Shigeru Miyamoto had a different solution – to widen gaming’s appeal. To do so, Nintendo had to break the research mould and sample non-gamers in their various forms. This meant Nintendo could widen their market. 110.61mn sales later, and the Nintendo Wii shows the value creation potential if you break sampling traditions.
A slow learner
It took Dietrich Mateschitz 10 years to earn his marketing degree. In the same time span, Mateschitz’s company Red Bull went from being launched in Austria to selling 300mn cans yearly. This was only possible as Mateschiz went to Thailand and tried an energy drink drunk by builders and lorry drivers. He brought the idea back to Europe, then made and sold it to young, fast-living professionals – a far cry from Mateschiz’s original sample profile.
Red Bull isn’t an anomaly. Peter Vandor’s work on migrant entrepreneurship shows consistently that cross-cultural experience stimulates creativity, so let’s ditch innovation briefs’ geographical boundaries.
A philosophy professor
Not letting research and marketing teams own sampling decisions is key to progress. Involve design. Ask sales. In fact, ask as many people unaware of the innovation you’re researching for their opinion. In doing so, you’ll stop sampling being designed in what Professor C Thi Nguyen calls an ‘epistemic bubble’. This is an information network where outside voices aren’t included. Often to the detriment of an outcome’s quality.
Next steps: cheese making & mining
How you apply these solutions and create more value in innovation research is spelt out within sampling’s etymology.
Some believe the word ‘sampling’ comes from the Dutch cheese making term ‘steekproef’. ‘Steeken’ means to cut. ‘Proeven’ means to taste or prove quality.
Others believe it comes from the German mining term ‘stitchprobe’. ‘Stitch’ means to dig or stab. ‘Probe’ means to try or test. So:
- Cut loose from sampling conventions
- Examples from Red Bull and Nintendo prove this is valuable
- Dig into fields outside of research for inspiration
- Don’t end a project and regret you didn’t try a new sampling approach. Yes, new things might fail. But regret is worse than failure
