p
EXPERIMENTS IN
NON-EXPERIMENTAL
SETTINGS
Strategy Experiments in Nonexperimental Settings: Challenges of Theory,
Inference, and Persuasion in Business Strategy" by Ron Adner & Daniel A. Levinthal
Presenter: Rida Zaman
rida.zaman@unive.it
Instructor: Professor Francesco Rullani
STRATEGY
Ca’ Foscari University of Venice | Venice School of Management | PhD Course: Strategy For Innovation Management
1
TODAY'S
AGENDA
Introduction
1
Problem Statement
2
key Concepts Introduced
3
Proposed Framework
4
The Four Quadrants
5
Implications for Inference
6
Role of Theory
7
Practical Insights
8
Conclusion
9
2
This article addresses the challenges faced by strategists when trying to
learn from "experiments" in real world business settings. Unlike scientific
experiments conducted in labs, strategy initiatives are often nonrepeatable
and require joint action among many stakeholders.
This complicates learning, inference, and persuasion three critical
components of effective strategy development.
Learning: Gaining understanding
Inference: Drawing meaning from information
Persuasion: Communicating that understanding effectively
INTRODUCTION
Example: The manager learned from employee behavior, inferred a better way of working, and
persuaded leadership to act on it.
Observation Concludes
Convince
3
Business Strategy vs. Experiments:
•Managers are told to test ideas like scientists.
•But real world strategy is messy, fast changing, and unpredictable.
•Actions can't be repeated the same way (non repeatability).
•Success often depends on others' support (joint action).
•So, it's hard to learn, prove cause-effect (inference), or gain support
(persuasion).
We need new ways to use theory for action, learning, and coalition-building.
PROBLEM
STATEMENT
4
Nonrepeatability:
Strategic actions often reshape the environment in ways that
prevent exact repetition.
⚬ Example: Coca-Cola revising its flagship soda formula
permanently alters customer expectations and brand
perception.
⚬ Example: Facebook rebranding to Meta impacts how
future initiatives are perceived.
KEY CONCEPTS
INTRODUCED
The authors introduce two important dimensions that differentiate
strategic experiments from traditional scientific experiments:
Joint Action:
Strategies typically require alignment and coordination with
multiple internal and external actors.
⚬ Example: Edison needed government permits, suppliers,
and investors for the Pearl Street power plant.
These dimensions highlight why strategy development must go
beyond simple trial and error and rely heavily on theory.
5
LAB BENCH
EXPERIMENTATION
STRATEGIC DEBATE
ACTING WITH ENDURING
CONCEQUENCES
JOINT STARTEGIC ACTION
The framework proposed by the authors is structured around two axes:
• Repeatability: Whether the same action can be repeated with consistent outcomes.
• Actor Type: Whether the strategy is driven by an individual or requires collective effort.
This creates four distinct quadrants for analyzing strategic initiatives.
PROPOSED
FRAMEWORK
INDIVIDUAL JOINT
REPEATABLE
NON - REPEATABLE
6
Individual & Repeatable
THE FOUR QUADRANTS
QUADRANT 2
Joint & Repeatable
QUADRANT 3
Individual & Nonrepeatable
QUADRANT 4
Joint & Nonrepeatable
QUADRANT 1
7
QUADRANT 1 - INDIVIDUAL & REPEATABLE
In this first quadrant, one person can
try something over and over again
without changing the environment.
This is like working in a lab or testing a
new product in its early stages. It’s easy
to learn here because nothing around
you changes.
A great example used in the article is
Edison testing 10,000 filaments for his
light bulb. Each time he failed, he
learned something new. But the testing
space didn’t change so it was a stable
environment for learning.
8
QUADRANT 2 - JOINT & REPEATABLE
In the second quadrant, you still have a stable
environment, but you need help from
others.
This could be internal meetings in a company
where people work together to design a
solution.
For example, when Edison worked on the Pearl
Street power station, he needed help from the
government, suppliers, and investors.
Even though the environment stayed mostly the
same, he had to build support and understanding
with others.
That’s where theory helped him explain his
ideas and bring everyone on board.
9
QUADRANT 3 - INDIVIDUAL & NONREPEATABLE
Now we move into trickier territory. In the
third quadrant, a single decision maker
makes a big move but the move changes the
environment so much that it can’t be
repeated.
Take Apple, for example. When they
launched the first iPhone, it completely
changed the smartphone market, what
customers expect, and how competitors
behave.
Even if they tried to “undo” that move,
they’d never go back to how things were.
That’s non repeatability in action.
10
QUADRANT 4 JOINT & NONREPEATABLE
Finally, we have the most complex case when
the strategy needs many people working
together, and it changes the environment so
much that it can’t be done again the same way.
A few examples from the article include Pfizer’s
failed inhalable insulin, which changed how the
pharmaceutical industry viewed similar projects.
Or Facebook’s move to Meta once they did that,
it set a new standard for tech platforms.
Another case is Better Place, an electric car
company with a battery-switching model that
failed and scared off other companies from
trying something similar.
11
So, what does this mean for how we learn, or
what we call inference?
In science labs, things are predictable because
conditions don’t change. You can assume that
the same input will give you the same result.
But in business, one strategic move can
completely change the game. The past might
not help you understand the future.
An example is A/B testing on platforms like
Airbnb. If users in one group interact with
those in another, the test results get mixed up
and it becomes hard to know what really
worked.
IMPLICATIONS
FOR INFERENCE
12
That’s why theory becomes so important.
It does two things:
• First, it helps you predict what might happen.
• Second, it helps you persuade others your
team, your investors, your partners.
Inside the company, theory helps people agree on
a plan. Outside, it helps you win support and trust.
ROLE OF
THEORY
13
LEARNINGS FROM
EACH CASE
DISCUSSED IN
ARTICLE
14
Company -Case Key Lesson Scenario Discussed
Coca-Cola – New
Coke
Nonrepeatability +
Loss of brand trust
Coca-Cola changed the formula of its iconic drink. The public
responded negatively, and even after reverting to the original, trust
and perception had already shifted.
Facebook – Meta
Rebranding
Irreversible identity
shift
Facebook rebranded as Meta to signal a focus on the metaverse.
However, the name change impacted how the world viewed the
company in an irreversible way.
Edison – Light Bulb
Individual +
Repeatable
experimentation
Edison independently tested thousands of filaments in a lab-like
setting. He failed repeatedly but learned with each test without
changing the external world.
Edison – Pearl
Street Station
Power Plant
Joint + Repeatable
experimentation
To launch the first commercial power plant, Edison needed
collaboration from governments, suppliers, and investors. The
challenge was persuading others to support a joint vision.
Microsoft / SAP –
Buggy Software
Update
Big brand = Big
reputational risk
When small companies release buggy updates, people are more
forgiving. But for major firms like Microsoft or SAP, such failures can
severely damage reputation.
Pfizer – Inhalable
Insulin
Failure discourages
future ecosystem
initiatives
Pfizer attempted to launch inhalable insulin. After it failed, other
companies became hesitant to pursue similar innovations, fearing
the same fate.
Project Better Place
– Battery Swapping
One failure can kill
an entire model
The company introduced a battery-swapping system for electric
cars. It failed to scale, and its failure discouraged others from
pursuing similar infrastructure-heavy ideas.
Michelin –
Repairable Run-Flat
Tires
Innovation alone ≠
Market success
Michelin launched innovative repairable tires. Despite the
technology being sound, the market didn’t accept it, showing that
innovation needs market validation too.
Pfizer / COVID-19
Vaccine
Scientific theory ≠
Public persuasion
Even though the COVID vaccine passed scientific tests, public
mistrust and conspiracy theories led to resistance, showing that
success also depends on effective communication and trust-
building.
TAKE AWAYS
WHAT ARE THE KEY TAKE AWAYS FROM THIS PAPERS:
1. First, know that big strategic actions are often irreversible you can’t just try again if they
go wrong.
2. Second, learning from experience is limited so using theory is key.
3. And third, persuasion is just as important as planning.
In the article, the authors talk about how difficult it is to restart belief and support after a
failed strategy. When companies “pivot,” they don’t just need a new idea they need to rebuild
shared purpose.
15
CONCLUSION
Strategic decision making in real life is very different from scientific experiments.
In business:
• You can’t always repeat actions.
• You often need others to make your plan work.
So we need theory to:
• Help us navigate (like a compass)
• Help us convince (like a megaphone)
Because strategists don’t just watch the world they help shape it.
16
THANK YOU
Rida Zaman
rida.zaman@unive.it
for your time and attention
17

Strategy Experiments in Nonexperimental Settings_ Challenges of Theory, Inference, and Persuasion in Business Strategy_.pdf

  • 1.
    p EXPERIMENTS IN NON-EXPERIMENTAL SETTINGS Strategy Experimentsin Nonexperimental Settings: Challenges of Theory, Inference, and Persuasion in Business Strategy" by Ron Adner & Daniel A. Levinthal Presenter: Rida Zaman [email protected] Instructor: Professor Francesco Rullani STRATEGY Ca’ Foscari University of Venice | Venice School of Management | PhD Course: Strategy For Innovation Management 1
  • 2.
    TODAY'S AGENDA Introduction 1 Problem Statement 2 key ConceptsIntroduced 3 Proposed Framework 4 The Four Quadrants 5 Implications for Inference 6 Role of Theory 7 Practical Insights 8 Conclusion 9 2
  • 3.
    This article addressesthe challenges faced by strategists when trying to learn from "experiments" in real world business settings. Unlike scientific experiments conducted in labs, strategy initiatives are often nonrepeatable and require joint action among many stakeholders. This complicates learning, inference, and persuasion three critical components of effective strategy development. Learning: Gaining understanding Inference: Drawing meaning from information Persuasion: Communicating that understanding effectively INTRODUCTION Example: The manager learned from employee behavior, inferred a better way of working, and persuaded leadership to act on it. Observation Concludes Convince 3
  • 4.
    Business Strategy vs.Experiments: •Managers are told to test ideas like scientists. •But real world strategy is messy, fast changing, and unpredictable. •Actions can't be repeated the same way (non repeatability). •Success often depends on others' support (joint action). •So, it's hard to learn, prove cause-effect (inference), or gain support (persuasion). We need new ways to use theory for action, learning, and coalition-building. PROBLEM STATEMENT 4
  • 5.
    Nonrepeatability: Strategic actions oftenreshape the environment in ways that prevent exact repetition. ⚬ Example: Coca-Cola revising its flagship soda formula permanently alters customer expectations and brand perception. ⚬ Example: Facebook rebranding to Meta impacts how future initiatives are perceived. KEY CONCEPTS INTRODUCED The authors introduce two important dimensions that differentiate strategic experiments from traditional scientific experiments: Joint Action: Strategies typically require alignment and coordination with multiple internal and external actors. ⚬ Example: Edison needed government permits, suppliers, and investors for the Pearl Street power plant. These dimensions highlight why strategy development must go beyond simple trial and error and rely heavily on theory. 5
  • 6.
    LAB BENCH EXPERIMENTATION STRATEGIC DEBATE ACTINGWITH ENDURING CONCEQUENCES JOINT STARTEGIC ACTION The framework proposed by the authors is structured around two axes: • Repeatability: Whether the same action can be repeated with consistent outcomes. • Actor Type: Whether the strategy is driven by an individual or requires collective effort. This creates four distinct quadrants for analyzing strategic initiatives. PROPOSED FRAMEWORK INDIVIDUAL JOINT REPEATABLE NON - REPEATABLE 6
  • 7.
    Individual & Repeatable THEFOUR QUADRANTS QUADRANT 2 Joint & Repeatable QUADRANT 3 Individual & Nonrepeatable QUADRANT 4 Joint & Nonrepeatable QUADRANT 1 7
  • 8.
    QUADRANT 1 -INDIVIDUAL & REPEATABLE In this first quadrant, one person can try something over and over again without changing the environment. This is like working in a lab or testing a new product in its early stages. It’s easy to learn here because nothing around you changes. A great example used in the article is Edison testing 10,000 filaments for his light bulb. Each time he failed, he learned something new. But the testing space didn’t change so it was a stable environment for learning. 8
  • 9.
    QUADRANT 2 -JOINT & REPEATABLE In the second quadrant, you still have a stable environment, but you need help from others. This could be internal meetings in a company where people work together to design a solution. For example, when Edison worked on the Pearl Street power station, he needed help from the government, suppliers, and investors. Even though the environment stayed mostly the same, he had to build support and understanding with others. That’s where theory helped him explain his ideas and bring everyone on board. 9
  • 10.
    QUADRANT 3 -INDIVIDUAL & NONREPEATABLE Now we move into trickier territory. In the third quadrant, a single decision maker makes a big move but the move changes the environment so much that it can’t be repeated. Take Apple, for example. When they launched the first iPhone, it completely changed the smartphone market, what customers expect, and how competitors behave. Even if they tried to “undo” that move, they’d never go back to how things were. That’s non repeatability in action. 10
  • 11.
    QUADRANT 4 JOINT& NONREPEATABLE Finally, we have the most complex case when the strategy needs many people working together, and it changes the environment so much that it can’t be done again the same way. A few examples from the article include Pfizer’s failed inhalable insulin, which changed how the pharmaceutical industry viewed similar projects. Or Facebook’s move to Meta once they did that, it set a new standard for tech platforms. Another case is Better Place, an electric car company with a battery-switching model that failed and scared off other companies from trying something similar. 11
  • 12.
    So, what doesthis mean for how we learn, or what we call inference? In science labs, things are predictable because conditions don’t change. You can assume that the same input will give you the same result. But in business, one strategic move can completely change the game. The past might not help you understand the future. An example is A/B testing on platforms like Airbnb. If users in one group interact with those in another, the test results get mixed up and it becomes hard to know what really worked. IMPLICATIONS FOR INFERENCE 12
  • 13.
    That’s why theorybecomes so important. It does two things: • First, it helps you predict what might happen. • Second, it helps you persuade others your team, your investors, your partners. Inside the company, theory helps people agree on a plan. Outside, it helps you win support and trust. ROLE OF THEORY 13
  • 14.
    LEARNINGS FROM EACH CASE DISCUSSEDIN ARTICLE 14 Company -Case Key Lesson Scenario Discussed Coca-Cola – New Coke Nonrepeatability + Loss of brand trust Coca-Cola changed the formula of its iconic drink. The public responded negatively, and even after reverting to the original, trust and perception had already shifted. Facebook – Meta Rebranding Irreversible identity shift Facebook rebranded as Meta to signal a focus on the metaverse. However, the name change impacted how the world viewed the company in an irreversible way. Edison – Light Bulb Individual + Repeatable experimentation Edison independently tested thousands of filaments in a lab-like setting. He failed repeatedly but learned with each test without changing the external world. Edison – Pearl Street Station Power Plant Joint + Repeatable experimentation To launch the first commercial power plant, Edison needed collaboration from governments, suppliers, and investors. The challenge was persuading others to support a joint vision. Microsoft / SAP – Buggy Software Update Big brand = Big reputational risk When small companies release buggy updates, people are more forgiving. But for major firms like Microsoft or SAP, such failures can severely damage reputation. Pfizer – Inhalable Insulin Failure discourages future ecosystem initiatives Pfizer attempted to launch inhalable insulin. After it failed, other companies became hesitant to pursue similar innovations, fearing the same fate. Project Better Place – Battery Swapping One failure can kill an entire model The company introduced a battery-swapping system for electric cars. It failed to scale, and its failure discouraged others from pursuing similar infrastructure-heavy ideas. Michelin – Repairable Run-Flat Tires Innovation alone ≠ Market success Michelin launched innovative repairable tires. Despite the technology being sound, the market didn’t accept it, showing that innovation needs market validation too. Pfizer / COVID-19 Vaccine Scientific theory ≠ Public persuasion Even though the COVID vaccine passed scientific tests, public mistrust and conspiracy theories led to resistance, showing that success also depends on effective communication and trust- building.
  • 15.
    TAKE AWAYS WHAT ARETHE KEY TAKE AWAYS FROM THIS PAPERS: 1. First, know that big strategic actions are often irreversible you can’t just try again if they go wrong. 2. Second, learning from experience is limited so using theory is key. 3. And third, persuasion is just as important as planning. In the article, the authors talk about how difficult it is to restart belief and support after a failed strategy. When companies “pivot,” they don’t just need a new idea they need to rebuild shared purpose. 15
  • 16.
    CONCLUSION Strategic decision makingin real life is very different from scientific experiments. In business: • You can’t always repeat actions. • You often need others to make your plan work. So we need theory to: • Help us navigate (like a compass) • Help us convince (like a megaphone) Because strategists don’t just watch the world they help shape it. 16
  • 17.