Here’s another good innovation article on Braden Kelley’s blog: Ethnography for Innovators by Chateau G Pato.
Here are the first paragraphs:
In our data-driven world, companies invest millions in surveys, focus groups, and A/B testing. Yet, these methods often only illuminate articulated needs—the problems people know they have and can describe. If you rely solely on these methods, you will, by definition, only produce incremental improvements on existing products.
The true gold standard of innovation—the breakthrough idea—lies in the unmet needs: the pervasive frictions, latent desires, or emotional compromises that people have simply grown used to and can no longer identify as problems. They are the invisible pain points that exist outside the structured environment of a corporate interview.
This gets to one thing I think holds a lot of companies back from doing meaningful innovation: the market research myth.
The myth is that managers believe answers on how to improve on existing products or create new products are as simple as doing market research, rank ordering the outcomes and executing on those.
It sounds logical. What could be wrong with that?
What’s wrong should be obvious. It doesn’t work. If it did, innovation would be easy and most companies would have more success in that regard.
What I find curious is how few people notice it doesn’t work and so the market research loop remains an unquestioned conventional business practice. When the last set of initiatives derived from the previous batch of market research flops, they go do start over. They do more market research to identify the next batch of initiatives and loop repeats itself.
It’s assumed that the initiatives or execution of the last batch was what was wrong, not the market research.
Here’s why I don’t think market research works.
Pato mentions one reason, the articulation fallacy. In a basic marketing course I learned about the difference between customers’ stated and revealed preferences.
People often say they want this or that (stated preference), but when given this or that, they still don’t buy it (revealed preference) and they can’t articulate why.
It’s usually because when the customers thought about a new feature in isolation, it sounded good. But, when faced with evaluating the value of that new feature relative to the other options available using their own money, it wasn’t worth it the extra cost or the risk to switch to something new, and they don’t think long or hard enough to understand this.
This is part of my business rule #1 of success: Have what customers want and customers don’t know what they want. In other words, you can’t count on customers to be able to tell you what they want. They will tell you what they think they want, but they don’t know what they actually want and they don’t know that they don’t know.
If they did know and could articulate it, again, innovation would be easy.
There are other things wrong with market research, too, like poor and biased analysis and overly simplistic conclusions.
I first noticed this while working with market researchers on pricing. According to the the market researchers, high price was the #1 problem to be fixed.
Managers listened to them and tried a number of price and promotional things to try to address it. None even budged the needle in sales or the market research. The same percentage of folks still complained about price even when they paid a much lower price than before.
After a few years of this, I asked the market researchers for the raw data that they used to base their claim on and analyzed it to discover that they had missed some important things.
Bad analysis: They assumed that the findings from a small disgruntled group of customers was representative of the whole customer population. They were not.
Missed: They missed that even in this small disgruntled group that these customers were really saying the value was not worth the price.
For example, one group said they thought their job was simple enough that they could DIY and save money. Market researchers saw the “save money” part and grouped it as a pricing problem. They ignored the part where the customer said they felt their job was simple enough to DIY, so the business folks were never aware of that.
Offering these customers discounts or lower prices hadn’t showed any signs of working.
My finding led to a few things that did work, like informing these customers that even if their job was simple, it’s also easy to mess up and cleaning up that mess can be costly. That worked better than any of the price offers because it addressed the value concern. It changed how those customers viewed the value from just the labor involved in completing the job to an investment to be made to avoid a costly cleanup.
Another problem with market research is that managers and market researchers believe it is a good substitute for putting products in market to predict a product’s success. It’s not. If it were, again, innovation would be easy.
The best innovators know that there is no substitute to the feedback you get from getting something in market, even if it isn’t the most perfectly refined product. There’s even a name for it: MVP or minimally viable product. It’s the best way to gauge interest and get start getting info on what will need to be improved.
The last problem I can think of with market research is that the MR folks usually don’t have any stake in the outcomes of the initiatives that stem from their findings, so they never have to consider that the initiatives failed because their findings were not right. They never have to course correct.
They will write-off that the initiatives or the execution and say those didn’t address the issue and keep repeating their same findings, as if it’s their job to point out the problem but not solve it.
If they had a stake in solving those problems and got to see firsthand what they personally believed wold solve the issue fail, that might cause them to consider that they missed something in their research and explore what that might be, instead of repeating the same old findings.