Near the end of his recent People I Mostly Admire podcast, Freakonomics co-author Levitt recalls a speech he gave about why businesses don’t use or appreciate natural experiments.

He describes a talk he gave…

I was talking about something that puzzles me, which is, why is it that natural experiment thinking is almost completely absent in the business world? Natural experiments or accidental experiments, the kind of research that I do, where you can’t run a randomized experiment, so you’ve got to go out and find other ways to get at causality. It’s really just an approach. It doesn’t take a lot of knowledge of statistics, it’s an attitude for finding answers to problems. And I think it would be really useful to businesses if they adopted it. But, I think in part because in high school or college or even in the M.B.A. programs, I don’t think we teach about the idea of what a natural experiment is and how to run it. It just doesn’t happen. So it was sort of a polemic about how business economists should latch onto the idea of analyzing their data through the lens of natural experiments.

I use natural experiments in the business world for my own work to great effect.

I agree that natural experiments are not appreciated.

Professional business managers aren’t interested in accidents. In their minds, business is driven forward by deliberate, smart strategy. Their deliberate, smart strategy. Or, at least, the flavor of the day management strategy that they are trying to emulate.

Natural experiments seems like a cop out to these folks.

They think, I didn’t go to Harvard or Yale just to learn to rely on happy accidents. Anybody can do that.

Bring up the idea of natural experiments with them, and they tend to get defensive and list all the successes they’ve had or heard about that they believe was smart strategy.

After all, some have told me, the iPhone wasn’t a happy accident. Steve Jobs has mythic status for smart strategy and they want to emulate it. They’ve cast themselves as the hero in that same type of story.

Then I start to list all the things that were precursors to the iPhone, including Apple’s first attempt at the personal digital assistant, the Apple Newton about 10 years prior to the first iPod, and then the iPod itself and the other products on the market that influenced the design and direction of the iPhone, including competing music players like Microsoft’s Zune & Blackberry phones.

In hindsight, it’s easy to attribute the the success of the iPhone as a lesson in smart strategy and dismiss the truer story of trial-and-failures and evolutionary processes and a bit of luck that led to it.

The winner always looks smart. And maybe they are.

But, dig a little deeper and you might find that the winners often leveraged natural experimentation to great effect. They may not have realized or it, or make the same attribution mistake when looking at their successes in hindsight.

I worked in a mature company where the leader came from a tech company that was growing while he was there. It was growing before he got there and stayed on the same trajectory until he left.

In hindsight, he attributed its growth during his tenure to things like an open office environment, instead of the company simply being on a natural growth trajectory after stumbling on a good value proposition.

So, he started implementing the open office environment at the mature company, hoping it would change its revenue trajectory to look like the growth company’s.

Spoiler-alert: It didn’t budge. It cost a lot of money and caused lots of strife in the workforce.

He quietly abandoned that about one-third through when it became abundantly clear to him that there wasn’t some magic elixir growth in having everyone be able to watch each other pick their noses all day long.