BoSON23 | Jim Morris | Designing and Running Customer Interviews that Work

Product Discovery Group © 2022 sfjmorris
ProductDiscoveryGroup.com
Customer Interviewing
Learn Quickly From Users to Achieve Your Outcomes
Jim Morris
Product Discovery Group
@sfjmorris

Utilize 25 Years of Experience to Create Lasting Change
$450 million IPO
$168 million sale

Tech, Finance, Energy
Coaching Product Organizations Across Industries
Consumer Digital Health

Why We Talk to Users

Save Money

Case Study - Spend to Learn Before Spending to Build
Before - High expense. Low learning. Zero traffic. After - Low expense. High learning. Ongoing traffic.

Stop Wasting Engineering Time

Case Study: IMVU
Eric Ries - Lean Startup The story of how Eric Ries
spent too much time and
money failing.
What could he have done
differently?

How Compelling Is The Idea?

The Right It
Alberto Savoia
former Google Director of
Engineering
Background Reading

“Skin in the Game”
Opinions are free
● Get a reaction
● Get an opinion
● Get a thoughtful response from the user
“Skin in the Game” costs something
● Get the user to actually give you their:
○ Real email address
○ Cell phone number
○ Home address
○ Credit card information
● Get the user to pay a ﬁnancial deposit ($50 to $1,000 to $25,000)
○ The higher the amount the higher the conﬁdence

Would-be Tesla owner spent
$1,000 to get on a waitlist for a car
that didn’t exist.
That is “Skin in the Game.”
It took almost 3 years to deliver.

Who Should We Interview?

Who Should We Interview?
● Someone you don’t know that ﬁts the description of your Target Customer
● Not your friends
● Not your coworkers
● Not your family
● Someone who has recently used the feature/product/service you have built
● Someone who might use the feature/product/service you would build
● It’s okay to interview users again if they have constructive feedback
● It’s okay to create a user group…as long as they are aligned to your vision

When We Talk To Users

The Three (3) Touchpoints of Modern Product Mgmt
● Gathering and Understanding Problems
● Verifying the Problem and Success Metric
● Testing Solutions with Users

Interview
Experts

First, Create a User Experience Map to
Find Problems and Opportunities

Identify Problems
and Opportunities
(and potential Solutions)

Second, Interview Experts to Validate
Your Intuition and Learn about More
Problems and Opportunities

How to Ask an Expert about Problems/Opportunities
● Ask expert to tell everything they know about the challenge at hand
● Where is the user experience map...
○ Wrong (Errors)
○ Needs additions (Additions)
● Ask expert to explain knowledge about customer decision making process
● Ask "Why" as your expert explains a certain area
● Say "Tell me more about it" to encourage expert to expand on topic

Jake Knapp - Sprint
Create a User Experience Map
● Chapter 4 (Start at the End)
● Chapter 5 (Map)
● Chapter 6 (Ask the Experts)
Jim’s Resources for User Experience Maps
● Miro board with slides and templates

Other Customer Touchpoints to Find Problems/Opptys
● Win / Loss analysis
● On-Site Visit / Day in a Life of a Customer
● App store reviews
● Sit in on customer calls (sales, customer success, support)
● Client listening tour
● Quantitative surveys

Interview
Business Buyers
and Users

Another Cautionary Tale

Case Study - Pivoting Your Strategy
Before - Roadmap of products and features After - Total focus on mobile compatibility

Don’t Build Your Product…
…Until You Validate
a Problem and a Success Metric

Opportunity Assessment

How to Interview with the Opportunity Assessment
NOTE: This technique works best in B2B products
● Verify EACH of the following with a potential customer:
○ Target Buyer (B2B)
○ Target Customer
○ Customer Problem
○ Business Objective
○ Baseline Metrics & Key Result
○ Constraints
● Ask what might be:
○ Wrong (Errors)
○ Incomplete (Omissions)
○ Added (Additions)

Marty Cagan - Inspired
Opportunity Assessment
● Business Objective
● Key Result (metric)
● Target Customer
● Customer Problem
Jim’s Resources for Opportunity Assessments
● Miro board with slides and templates

Solution Tests
with Users

Lean UX is about reducing risk
— Article by Drew McKinney, diagram attributed to Pivotal Labs and Janice Fraser

Making a Great Solution Test is…
More Powerful Than Being a Great
Interviewer

Design Prototype Interview
(quicker than writing code…
remember the Cautionary Tale)

Example:
Testing Multiple Ways to Improve Zoom
Breakout Rooms

Quickly Test Multiple Ideas
Looking at an list of many ideas…
…how can we use rapid discovery to ﬁnd which
items will produce the most value for users and
work on them ﬁrst?

“Zoom now offers
advanced features to
manage the success
and health of
Breakout Rooms.”

“Zoom now offers the
ability to Listen In to a
Breakout Room without
having to enter it.”

“New feature. See who
has video turned on or
off in Breakout Rooms
without having to enter
them.”

“New feature. For
Breakout Rooms, Zoom
shows you which rooms
are the quietest and
which rooms have the
most people talking.”

Video Example:
Testing Multiple Ways to Improve
Classroom Technology

Onsite in a
classroom in
San Francisco
Interviewing
teachers

What We Will Learn In The Upcoming Interview
Evaluating Ideas
● Which feature is the user actually excited about?
Notes to Setup the Interview
● User is in control of the prototype
● Prototype is VERY simple

Don’t Settle for Blah Feedback

Q & A

Modern Product Development
Instead of focusing on one solution too early…
…test multiple ﬂows. Create a compare and contrast moment
that reveals the user’s choice (if any)
…stretch the creativity and imagination of your team by taking
smart risks

No Choices →
Conﬁrmation Bias
Choices →
Open Minded
or

How do you know
where to start?

Solution Test Example:
Improve Zoom Breakout Rooms

Jim Does a “Bad” Interview

Don’t Say These Things

Customer Interviewing Don’ts
● Don’t talk so much
● Don't narrate the on-screen options to the user
● Don't ﬁll the silence
● Avoid helping users…wait…wait…wait
● Don't say…
○ "What do you see?"
○ "Describe this screen"
○ "What do you think about this screen?”
● Avoid saying "What's missing here?"
● Don't ask users to "Read an email"

Jim Does a “Good” Interview

Everyone Open the Interview Script
pdg.info/script

What Were the Differences?

What Were the Differences?
In the 2nd interview:
● User controls the prototype
● User talks more → I talk less → Less chance to introduce bias
● There are setup screens to give the user context
● We ask the user to tell us story relevant to the pain point we are testing
○ This authenticates the “Relevant” part of the “Relevant Stranger”
● We ask a survey question to record a vote from the user (NPS, Disappointment)
● We ask for “Skin in the Game”
● We are hypothesis-focused

Customer Interviewing Dos
● If the user becomes silent
○ Ask them: “Go ahead and think out loud”
● If the user gets stuck
○ Remind them of the prompt
○ Ask them: “What would you do next?”
● If the user asks you a question about how something works
○ Ask them how they think it should work
● Discourage users from talking about others (hearsay)

Follow a Script
(sort of)

pdg.info/script → Make the User Comfortable

pdg.info/script → Have user control prototype

pdg.info/prototype → Clickable Prototype

pdg.info/script → Collect a relevant story

pdg.info/script → Prompt the user to start

pdg.info/script → Hypotheses based interview

pdg.info/script → Return to beginning
→ Guide user to Option 2
(in Figma press “r” to go back to options)
→ Then Option 3
→ Then Compare and Contrast

pdg.info/script → Find the User’s Preference

pdg.info/script → Ask for “Skin in the Game”

Now, Everyone Will Interview Each Other

Split Into Into Groups of Two
One person is the interviewer. The other is the user tester.
● Play “Rock, Paper, Scissors” to see who wins
● The winner becomes the interviewer
● The “loser” becomes the user tester
The interviewer uses the Script to lead the interview → pdg.info/script
The user tester acts like a professor moderating LOTS of breakout rooms
You get 10 minutes to interview the user tester. Then switch roles.
Prototype (included in the Interviewer script): pdg.info/prototype
Customer Interviewing Practice

Setup
Interviewer → Script
pdg.info/script
Tester → Prototype
pdg.info/prototype

● What’s it like leading an interview?
● What did you feel comfortable at?
● What made you feel nervous?
● What’s the difference between good user interviewing and great user
interviewing?
● What was it like to be the user tester?
Retrospective: Customer Interviewing

Tips to Remember
● Use a script. Use a clickable prototype.
● Make the user comfortable
● Have the user control the prototype on their screen
● Collect a relevant user story up front
○ End the interview immediately if the user is not relevant
● Collect background information as you go
● Let the hypotheses guide your questions
● Remind the user about the prompt → What would you do next?
● Follow the user as they explore the prototype
● Get the user’s preference at the end of the interview
● Get “Skin in the Game”

Resources: The Solution Test Interview Explained

Customer Interviewing Recap

Conduct
Solution Tests
with Users

Do you need a professional UX researcher
to interview customers?
No

How many customers/prospects should
you meet per week?
at least 3

Jim’s Favorite Product Management Tools
Virtual Whiteboard → Miro.com
Prototyping/Design → Figma
Remote Interviewing → Zoom
Self-service User Recruiting (US + other countries) → UserInterviews.com
Screenshots/Quick Designs/Visual Annotations → SnagIt
Collaboration → Google Drive + Slack + Zoom

Questions?

linkedin.com/in/jimmorrisstanford
@sfjmorris @sfjmorris
productdiscoverygroup.com/subscribe
productdiscoverygroup.com/learn
Stay in Touch

Write Feedback about this Presentation:
pdg.info/survey

BoSON23 | Jim Morris | Designing and Running Customer Interviews that Work

More Related Content

Similar to BoSON23 | Jim Morris | Designing and Running Customer Interviews that Work

More from Business of Software Conference

Recently uploaded

BoSON23 | Jim Morris | Designing and Running Customer Interviews that Work