“If your experiment needs statistics, you ought to have done a better experiment.” This wise quote from British physicist & Nobel prize winner Lord Ernest Rutherford applies perfectly to what we do here at HYVE. Experimentation is at the heart of design thinking and is customer-centric. It is essential to test assumptions, acquire new knowledge and de-risk the innovation process. Experimentation relies on observation and on questioning, it is thus qualitative. It validates or refutes assumptions, highlights areas of improvement and helps innovators know whether to pivot or persevere.
Statistics on the other hand are often perceived as hard facts, they influence decision-making (too much) and are taken for granted. Very rarely do the people who use statistics question how these numbers have been obtained, what they mean, or how they relate to each other. Too often these numbers are taken in isolation and one stat suffices to convince (wrongly) someone to act. In the field of user-centred research, numbers do not eliminate uncertainties, they simply give the false impression of it.
What is an insight?
Building a strong relationship with your customer begins with having an exceptionally good understanding of him or her. Similarly, marketing breakthroughs and bestselling innovations are often based on solid customer understanding, aka “so-called” customer insights. If not the most recent one, Apple’s iPod is an excellent example to show the importance of such customer insights. The iPod was not the first digital music player in the industry when it launched, but it was the only one who offered customers what they needed, that is the largest number of songs in the smallest possible device.
Although customer insights are crucial, they are difficult to uncover and also to define. Everyone from agencies to companies and startups has a different understanding of what an insight is. At HYVE we are 100% aligned with the definition of Mohanbir Sawhney, Professor of Technology at the McCormick Foundation and Director of the Center for Research in Tech & Innovation of the Kellogg School of Management: a customer insight is “a fresh understanding, but not obvious, about consumers’ beliefs, values, desires, motives, emotions and needs, that can be translated into competitive advantage.” This definition describes several characteristics of an insight but the most important one reads as follows: insights rarely come from quantitative research!
As Steve Bennett, former CEO of Intuit, observed in a speech at Northwestern University’s Kellogg School of Management years ago: “I have not seen a single insight come out of quantitative market research.” However, companies continue to spend significant amounts of money on strict surveys and sophisticated quantitative techniques “to uncover insights”. It is also a question that we hear a lot from our clients: “how many people said that or felt this way?” Experience proves that breakthrough insights can be born from a single comment. Professor Mohanbir Sawhney explains the difference between qualitative and quantitative research: “A consumer insight never comes from quantitative research so don’t look for it in your surveys. Qualitative research and quantitative research need to go hand in hand for true customer understanding. Qualitative research generates insights, while quantitative research validates insights. You need both research traditions, and in the right sequence.”
What should I (not) use quantitative research for?
So quantitative research can be used to validate insights, from quantifying personas in a specific market to checking on the validity of an insight on a national scale. However, too often research briefs ask for quantitative research early in the innovation process when insights haven’t yet been uncovered. Quantitative research can also take the form of 54 four-hour workshops in 3 cities under 3 weeks or can be forced into pricing questions that customers cannot answer. The former is a waste of money and energy because you start hearing the same thing over and over after 2 workshops with 16 people, and the latter is a difficult question to be answered by business analysts, let alone for customers!
Let’s do the experiment together. Please ask yourself: how much would I be willing to pay for a flying car? The most likely answer is: you don’t know. You might be thinking “I already pay X for leasing my car and Y for national flights, so maybe I would be willing to pay something in between?”. You’re using the anchoring technique to try and evaluate the price of something that does not exist and that you cannot compare like for like to another product. And that’s fine! But imagine you are being forced to answer this question by choosing among a drop-down list of prices, which one will it be? €200 a week? €1,000 a month? €50,000 as a one-off? “No but give me a number.” Chance is by that point you will make up any number to please your interviewer or cross the lowest number on the list of the online survey that you are filling in.
Last but not least, we often hear our clients say “management wants numbers”. This is perfectly understandable, but there are different ways to present results to management that do not necessarily include numbers or that can make a bigger emotional impact. The power of storytelling, the voice of a lead user or influencer on social media, bringing customers in the boardroom virtually or physically are just a few examples of this.
As a conclusion, a mix of methods is the best way to go: qualitative and quantitative, passive and active, individual and collective, online and offline. Start with qualitative insights, end with quantitative validation.
We’d love to hear your point of view and questions on the topic, but also your qualitative stories, so don’t hesitate to get in touch. As researchers we love to listen.
Aurelie is Innovation Strategist at HYVE. Aurelie has worked several years in insights and innovation in London before joining HYVE and taking over larger, international innovation projects.
Nayeli is Senior Innovation Researcher. With her B. Sc. in Cognitive Psychology and M.Sc. in Consumer Science she has “Eagle Eyes” when it comes to identifying and delivering solid, impactful insights.