SlideShare a Scribd company logo
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
@shanelle_mullin
We’ll cover…
1. How to conduct conversion research to find smarter A/B test ideas.
2. How to prioritize your A/B test ideas in a meaningful way.
3. What a high velocity testing program is and why it works.
4. Common high velocity testing program mistakes to avoid.
5. How to increase your testing velocity responsibly.
How to Run Smarter A/B Tests, Faster
“You don’t have generic problems, you
have specific problems.”
~Peep Laja
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
1. Relevancy
Does the page’s content meet user expectation?
Does the page’s design meet user expectation?
How can it match what they expect / want even more?
How to Run Smarter A/B Tests, Faster
2. Clarity
Is the content on this page as clear as possible?
Is the value prop on this page as clear as possible?
How can we make it clearer?
How to Run Smarter A/B Tests, Faster
3. Motivation
What’s motivating the visitor to take action?
Is it clear, believable, timely and persuasive?
How can we increase motivation even more?
How to Run Smarter A/B Tests, Faster
4. Friction
What is causing doubts, hesitations and uncertainties?
What makes the process difficult?
How can we simplify the process?
How to Run Smarter A/B Tests, Faster
5. Distraction
What’s not helping the user take action?
What’s drawing attention?
Is everything that draws attention really necessary?
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Cross-Browser Testing
Audience > Technology > Browser & OS
Switch from the default “Data” view to the “Comparison” view.
How to Run Smarter A/B Tests, Faster
Cross-Browser Testing
Always compare within the same browser family.
How to Run Smarter A/B Tests, Faster
Cross-Device Testing
Audience > Mobile > Devices
Switch from the default “Data” view to the “Comparison” view.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Speed Analysis
Page Load Time: Seconds until every element on the
page is done loading.
Page Interactive Time: Seconds until the site is usable.
How to Run Smarter A/B Tests, Faster
Speed Analysis
Your site loads in 3 seconds or less.
Your site loads in 4 to 7 seconds.
Your site loads in 7 to 10 seconds (or more).
How to Run Smarter A/B Tests, Faster
Speed Analysis
Behavior > Site Speed > Page Timings
Switch from the default “Data” view to the “Comparison” view.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Google Analytics Health Check
1. Are you collecting all of the data you need?
2. Can you trust the data you are collecting?
3. Is anything broken or tracking / reporting
incorrectly? Why?
How to Run Smarter A/B Tests, Faster
Common Google Analytics Mistakes
1. Pages are missing.
2. Data is different in the shopping cart tool.
3. Cross-domain tracking isn’t working.
How to Run Smarter A/B Tests, Faster
Full Health Check Guide
bit.ly/gahealthcheck
How to Run Smarter A/B Tests, Faster
Start with the Highest Value
1. High volume, low value pages.
2. Low volume, high value pages.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Finding Broken Links
How to Run Smarter A/B Tests, Faster
Finding Broken Links
How to Run Smarter A/B Tests, Faster
Auditing Your Funnel
The further down the funnel, the smaller the
necessary impact.
How to Run Smarter A/B Tests, Faster
Auditing Your Funnel
How to Run Smarter A/B Tests, Faster
Auditing Your Funnel
How to Run Smarter A/B Tests, Faster
Exploring Internal Search
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Segmentation Is the Secret
bit.ly/gasegmentation
How to Run Smarter A/B Tests, Faster
Click Maps
1. This info is largely available in Google Analytics via
“Navigation Summary”.
2. See when people are clicking on non-links.
How to Run Smarter A/B Tests, Faster
Scroll Maps
1. Better prioritize your content and messaging.
2. Drastic colour changes mean a need for visual
cues… likely indicate false bottom.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Session Replays
1. Real users with real money.
2. Where do they pause? Where do they give up?
3. Use this to gather more insight on poor performers.
How to Run Smarter A/B Tests, Faster
Form Analytics
1. Which form fields cause the most error messages?
2. Which do people hesitate to fill?
3. Which do people leave empty, even if required?
How to Run Smarter A/B Tests, Faster
On-Site Surveys
Exit surveys.
On-page surveys.
How to Run Smarter A/B Tests, Faster
On-Site Survey Questions
1. What was the purpose of your visit to our site?
2. Were you able to complete your task?
3. If not, why?
4. Is there anything holding you back?
5. Do you have unanswered questions?
Focus on actionable data, on the friction.
How to Run Smarter A/B Tests, Faster
On-Site Survey Options
1 open-ended question.
Y/N and then an explanation post-answer.
How to Run Smarter A/B Tests, Faster
On-Site Survey Guide
bit.ly/onsitesurveys
How to Run Smarter A/B Tests, Faster
Customer Surveys
1. First time buyers.
2. No previous relationship with you.
3. Try for ~200 responses.
How to Run Smarter A/B Tests, Faster
Customer Survey Questions
1. Who are you?
2. What are you using this product for?
3. What problem does it solve for you?
4. What do you like about the product most?
5. Did you consider any alternatives?
6. Why did you choose us over the competition?
7. What’s the one thing that nearly stopped you from buying?
8. What was your biggest concern or fear about us?
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
User Testing Tasks
1. A specific task.
2. A broad task.
3. Funnel completion.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Constant Ideation
1. Continue conducting conversion research.
2. Involve the entire team.
3. Focus on the entire funnel (acquisition to referral).
How to Run Smarter A/B Tests, Faster
“If you come to me with an idea and it’s not
live in 2 weeks, it’s not because it’s a bad
idea. It’s because I have better things to test.”
~Pauline Marol
How to Run Smarter A/B Tests, Faster
The Prioritization Problem
1. Not objective enough.
2. Not data-driven enough.
3. Not easily customized.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
Benefits of PXL
1. Makes “potential” more objective.
2. Makes “ease” more objective.
3. Helps create a data-driven culture.
4. Easy to customize.
How to Run Smarter A/B Tests, Faster
Prioritization Spreadsheet
bit.ly/pxlmodel
How to Run Smarter A/B Tests, Faster
Better Management
1. Review KPIs.
2. Review tests launched (and not launched).
3. Review insights gained.
4. Choose tests for the upcoming week.
5. Discuss resources (or lack thereof).
How to Run Smarter A/B Tests, Faster
Archive (Or Live to Regret It)
1. Avoid repeating tests by mistake.
2. Easily communicate wins and insights.
3. Improve the quality of future tests.
4. Create an insights > wins culture.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
High Velocity Mistakes
1. Without buy-in and a testing culture, you’re
doomed.
2. Calling it too soon and non-representative samples.
3. Letting quality and effectiveness slip.
How to Run Smarter A/B Tests, Faster
Testing Capacity
(52 / Avg. Req. Test Duration) x Number of Testable Pages
(52 / 2) x 10 = 260
How to Run Smarter A/B Tests, Faster
Measuring Velocity
1. How well are you trending towards achieving your
capacity?
2. Trends are what matters… are you getting closer?
3. Chart it.
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
How to Run Smarter A/B Tests, Faster
@shanelle_mullin
1. Conduct proper conversion research.
2. Come up with ideas constantly.
3. Prioritize with data.
4. Manage the process.
5. Archive your learnings.
6. Achieve your testing capacity.
7. Stop listening to CRO BS you read online.
How to Run Smarter A/B Tests, Faster
Resources
Optimization Training Program
bit.ly/cxlinstitute
Original UX Research
bit.ly/cxlresearch
How to Run Smarter A/B Tests, Faster
Questions?
Ask any time via email: shanelle@conversionxl.com

More Related Content

What's hot (20)

PPTX
Ab testing 101
Ashish Dua
 
PPTX
Data-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision Making
indeedeng
 
PDF
Weapons of Math Instruction: Evolving from Data0-Driven to Science-Driven
indeedeng
 
PPTX
Anyone Can Write User Stories. It's the (Shared) Understanding That's Important
Kent McDonald
 
PDF
Optimizely Workshop: Take Action on Results with Statistics
Optimizely
 
PPTX
Content Testing & Optimization (CTO) Webinar, April 29 2010
One to One
 
PPTX
eMetrics London - The AB Testing Hype Cycle
Craig Sullivan
 
PDF
Talks@Coursera - A/B Testing @ Internet Scale
courseratalks
 
PDF
4 Steps Toward Scientific A/B Testing
Janessa Lantz
 
PPTX
How to stay relevant - An evolutionary talk for testers
Augusto Evangelisti
 
PDF
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
indeedeng
 
PPTX
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Craig Sullivan
 
PDF
Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...
MeetMagentoNY2014
 
PPT
Patterns in Testing and a claim - iCheckWebsite
Anand Ramdeo
 
PPT
Agile Testing
Anand Ramdeo
 
PPTX
Why do my AB tests suck? measurecamp
Craig Sullivan
 
PDF
Meet Magento Belarus - Viacheslav Kravchuk
Amasty
 
PPTX
How to Test Anything
James Thomas
 
PPTX
Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Craig Sullivan
 
PDF
Workshop 6: Build Your Organization's Optimization Culture
Optimizely
 
Ab testing 101
Ashish Dua
 
Data-Driven off a Cliff: Anti-Patterns in Evidence-Based Decision Making
indeedeng
 
Weapons of Math Instruction: Evolving from Data0-Driven to Science-Driven
indeedeng
 
Anyone Can Write User Stories. It's the (Shared) Understanding That's Important
Kent McDonald
 
Optimizely Workshop: Take Action on Results with Statistics
Optimizely
 
Content Testing & Optimization (CTO) Webinar, April 29 2010
One to One
 
eMetrics London - The AB Testing Hype Cycle
Craig Sullivan
 
Talks@Coursera - A/B Testing @ Internet Scale
courseratalks
 
4 Steps Toward Scientific A/B Testing
Janessa Lantz
 
How to stay relevant - An evolutionary talk for testers
Augusto Evangelisti
 
Indeed Engineering and The Lead Developer Present: Tech Leadership and Manage...
indeedeng
 
Digital Impact 2014 - Oh Boy These AB tests sure look like Bullshit to me
Craig Sullivan
 
Conversion rate optimisation. What's realluy proved to matter? Viacheslav kra...
MeetMagentoNY2014
 
Patterns in Testing and a claim - iCheckWebsite
Anand Ramdeo
 
Agile Testing
Anand Ramdeo
 
Why do my AB tests suck? measurecamp
Craig Sullivan
 
Meet Magento Belarus - Viacheslav Kravchuk
Amasty
 
How to Test Anything
James Thomas
 
Mobile presentation - Sydney Online Retailer - 26 Sep 2011
Craig Sullivan
 
Workshop 6: Build Your Organization's Optimization Culture
Optimizely
 

Viewers also liked (10)

PDF
Scoopit's Content Marketing Grader for ROI
Scoop.it
 
PPTX
Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...
Orbit Media Studios
 
PPTX
Applied Analytics: Insights and Actions from 13 Google Analytics Reports
Orbit Media Studios
 
PPTX
Advanced Content Marketing & SEO: How to Publish More and Rank Higher
Andy Crestodina
 
PPTX
Content Strategy at the Big Ooga!
Andy Crestodina
 
PDF
Clickbait: A Guide To Writing Un-Ignorable Headlines
Venngage
 
PDF
Secrets of the Most Successful Bloggers
Barry Feldman
 
PPTX
Intro to Mozcon 2016
Rand Fishkin
 
PDF
2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders
Bryan Kramer
 
PDF
B2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North America
Content Marketing Institute
 
Scoopit's Content Marketing Grader for ROI
Scoop.it
 
Content Jam: Fortune and Glory, How to Make Friends, Rank High and Get Famous...
Orbit Media Studios
 
Applied Analytics: Insights and Actions from 13 Google Analytics Reports
Orbit Media Studios
 
Advanced Content Marketing & SEO: How to Publish More and Rank Higher
Andy Crestodina
 
Content Strategy at the Big Ooga!
Andy Crestodina
 
Clickbait: A Guide To Writing Un-Ignorable Headlines
Venngage
 
Secrets of the Most Successful Bloggers
Barry Feldman
 
Intro to Mozcon 2016
Rand Fishkin
 
2017 Social Media & Content Marketing Predictions from 70 Marketing Leaders
Bryan Kramer
 
B2B Content Marketing 2017 - Benchmarks, Budgets & Trends - North America
Content Marketing Institute
 
Ad

Similar to How to Run Smarter A/B Tests, Faster (20)

PDF
What is AB testing, and why do you need it.pdf
iM4U Digital Marketing Agency
 
PDF
Definition of A/B testing and Case Studies by Optimizely
RusseWeb
 
PDF
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Future Insights
 
PDF
The Power of A/B Testing
Alexandre Pallota
 
PDF
A/B Testing: Fine-Tune Your Ads for Optimal Results
Ennoble Technologies
 
PDF
Analytics Academy 2017 Presentation Slides
HarvardComms
 
PDF
Why A/B testing is probably damaging your business (and what you can do about...
Steven Kent
 
PDF
Growing Your Business Through Experimentation
Hiten Shah
 
PPTX
AB testing.pptx
Mugabo4
 
PDF
How to Successfully Run Your First Website A/B Test
Kissmetrics on SlideShare
 
PDF
Data-Driven UI/UX Design with A/B Testing
Jack Nguyen (Hung Tien)
 
PPTX
Conversion Optimization Willa Fogarty
Willa Fogarty
 
PPTX
A/B Testing Best Practices - Do's and Don'ts
Ramkumar Ravichandran
 
PDF
Introduction to ab_testing_for_marketing_optimization
Turisticae
 
PPTX
abtesting-Updated
Sai krishna pramod Mandali
 
PPTX
Interact London - 21 Oct 2015 - Scaling Stupidity
Craig Sullivan
 
PPTX
Hong Kong Web Analytics Wednesday #10
Hong Kong Web Analytics
 
PDF
Qualaroo & Optimizely Webinar - 5 Steps to Your First A/B Testing Win
Optimizely
 
PDF
Conversion Rate Optimisation Guide
C.Y Wong
 
What is AB testing, and why do you need it.pdf
iM4U Digital Marketing Agency
 
Definition of A/B testing and Case Studies by Optimizely
RusseWeb
 
Always Be Testing - Learn from Every A/B Test (Hiten Shah)
Future Insights
 
The Power of A/B Testing
Alexandre Pallota
 
A/B Testing: Fine-Tune Your Ads for Optimal Results
Ennoble Technologies
 
Analytics Academy 2017 Presentation Slides
HarvardComms
 
Why A/B testing is probably damaging your business (and what you can do about...
Steven Kent
 
Growing Your Business Through Experimentation
Hiten Shah
 
AB testing.pptx
Mugabo4
 
How to Successfully Run Your First Website A/B Test
Kissmetrics on SlideShare
 
Data-Driven UI/UX Design with A/B Testing
Jack Nguyen (Hung Tien)
 
Conversion Optimization Willa Fogarty
Willa Fogarty
 
A/B Testing Best Practices - Do's and Don'ts
Ramkumar Ravichandran
 
Introduction to ab_testing_for_marketing_optimization
Turisticae
 
abtesting-Updated
Sai krishna pramod Mandali
 
Interact London - 21 Oct 2015 - Scaling Stupidity
Craig Sullivan
 
Hong Kong Web Analytics Wednesday #10
Hong Kong Web Analytics
 
Qualaroo & Optimizely Webinar - 5 Steps to Your First A/B Testing Win
Optimizely
 
Conversion Rate Optimisation Guide
C.Y Wong
 
Ad

Recently uploaded (20)

PDF
Digital Marketing Trends to Watch in 2025
thedigitalfoot
 
PPTX
DISSERTATION For management students.pptx
ganesh831732
 
PPT
Mastering Microsoft PowerPoint.ppt......
satsahibsatshib9
 
PDF
The AI-Powered Paid Media Playbook | Amsive Webinar
Amsive
 
PDF
OLALA COMMUNICATION _CREDENTIAL (compressed).pdf
ThinNgc37
 
PDF
Digital Marketing Landscape for Beginners (Nepali Case Study Included) – Day ...
Dipendra Poudel | The Digital Dipendra
 
PDF
Why Digital Marketing is the Future of Business Growth
Elysium Aviation Academy
 
PDF
Defining International Journalism in the Modern Age S Vijay Kumar Internatio...
SanskarTiwari20
 
PPTX
Top 10 Social Media Strategies to Boost Your Digital Marketing in 2024
hrithiksrivastava683
 
PDF
Becoming a Better You: How to Discover a Better Version of Yourself - Jamie T...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
PDF
The Early-Stage Growth Hack You’re Overlooking.pdf
AminaSeigell
 
PDF
First Thought Agency - all you need to know about us!
knownsense59
 
PPTX
Paid Search vs. Paid Social. Why You Should Combine
Higher Education Marketing
 
PDF
Partipost Vietnam Credential Deck (Influencer Marketing Agency)
scarletnguyen9
 
PDF
Follow for exclusive insights and conversations happening now on Threads.pdf
Rich Vibes Publication
 
PDF
How to Maximise Social Media Benefits with Digitale.pdf
Digitale
 
PDF
Unfiltered. Real. Relatable. Rich Vibes Publication on Medium
Rich Vibes Publication
 
PPTX
SXSW_ Cultural Intelligence Overview.pptx
ericabuchman1
 
PPTX
Presentation - Marketing Agency Benefits.pptx
khadijaaaism
 
PPTX
What Branding looks like, by: Cayancela Sánchez Jairo
Jairo Cayancela Sánchez
 
Digital Marketing Trends to Watch in 2025
thedigitalfoot
 
DISSERTATION For management students.pptx
ganesh831732
 
Mastering Microsoft PowerPoint.ppt......
satsahibsatshib9
 
The AI-Powered Paid Media Playbook | Amsive Webinar
Amsive
 
OLALA COMMUNICATION _CREDENTIAL (compressed).pdf
ThinNgc37
 
Digital Marketing Landscape for Beginners (Nepali Case Study Included) – Day ...
Dipendra Poudel | The Digital Dipendra
 
Why Digital Marketing is the Future of Business Growth
Elysium Aviation Academy
 
Defining International Journalism in the Modern Age S Vijay Kumar Internatio...
SanskarTiwari20
 
Top 10 Social Media Strategies to Boost Your Digital Marketing in 2024
hrithiksrivastava683
 
Becoming a Better You: How to Discover a Better Version of Yourself - Jamie T...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions
 
The Early-Stage Growth Hack You’re Overlooking.pdf
AminaSeigell
 
First Thought Agency - all you need to know about us!
knownsense59
 
Paid Search vs. Paid Social. Why You Should Combine
Higher Education Marketing
 
Partipost Vietnam Credential Deck (Influencer Marketing Agency)
scarletnguyen9
 
Follow for exclusive insights and conversations happening now on Threads.pdf
Rich Vibes Publication
 
How to Maximise Social Media Benefits with Digitale.pdf
Digitale
 
Unfiltered. Real. Relatable. Rich Vibes Publication on Medium
Rich Vibes Publication
 
SXSW_ Cultural Intelligence Overview.pptx
ericabuchman1
 
Presentation - Marketing Agency Benefits.pptx
khadijaaaism
 
What Branding looks like, by: Cayancela Sánchez Jairo
Jairo Cayancela Sánchez
 

How to Run Smarter A/B Tests, Faster

  • 1. How to Run Smarter A/B Tests, Faster How to Run Smarter A/B Tests, Faster @shanelle_mullin We’ll cover… 1. How to conduct conversion research to find smarter A/B test ideas. 2. How to prioritize your A/B test ideas in a meaningful way. 3. What a high velocity testing program is and why it works. 4. Common high velocity testing program mistakes to avoid. 5. How to increase your testing velocity responsibly.
  • 2. How to Run Smarter A/B Tests, Faster “You don’t have generic problems, you have specific problems.” ~Peep Laja
  • 3. How to Run Smarter A/B Tests, Faster
  • 6. How to Run Smarter A/B Tests, Faster 1. Relevancy Does the page’s content meet user expectation? Does the page’s design meet user expectation? How can it match what they expect / want even more?
  • 7. How to Run Smarter A/B Tests, Faster 2. Clarity Is the content on this page as clear as possible? Is the value prop on this page as clear as possible? How can we make it clearer?
  • 8. How to Run Smarter A/B Tests, Faster 3. Motivation What’s motivating the visitor to take action? Is it clear, believable, timely and persuasive? How can we increase motivation even more?
  • 9. How to Run Smarter A/B Tests, Faster 4. Friction What is causing doubts, hesitations and uncertainties? What makes the process difficult? How can we simplify the process?
  • 10. How to Run Smarter A/B Tests, Faster 5. Distraction What’s not helping the user take action? What’s drawing attention? Is everything that draws attention really necessary?
  • 13. How to Run Smarter A/B Tests, Faster Cross-Browser Testing Audience > Technology > Browser & OS Switch from the default “Data” view to the “Comparison” view.
  • 14. How to Run Smarter A/B Tests, Faster Cross-Browser Testing Always compare within the same browser family.
  • 15. How to Run Smarter A/B Tests, Faster Cross-Device Testing Audience > Mobile > Devices Switch from the default “Data” view to the “Comparison” view.
  • 17. How to Run Smarter A/B Tests, Faster Speed Analysis Page Load Time: Seconds until every element on the page is done loading. Page Interactive Time: Seconds until the site is usable.
  • 18. How to Run Smarter A/B Tests, Faster Speed Analysis Your site loads in 3 seconds or less. Your site loads in 4 to 7 seconds. Your site loads in 7 to 10 seconds (or more).
  • 19. How to Run Smarter A/B Tests, Faster Speed Analysis Behavior > Site Speed > Page Timings Switch from the default “Data” view to the “Comparison” view.
  • 23. How to Run Smarter A/B Tests, Faster Google Analytics Health Check 1. Are you collecting all of the data you need? 2. Can you trust the data you are collecting? 3. Is anything broken or tracking / reporting incorrectly? Why?
  • 24. How to Run Smarter A/B Tests, Faster Common Google Analytics Mistakes 1. Pages are missing. 2. Data is different in the shopping cart tool. 3. Cross-domain tracking isn’t working.
  • 25. How to Run Smarter A/B Tests, Faster Full Health Check Guide bit.ly/gahealthcheck
  • 26. How to Run Smarter A/B Tests, Faster Start with the Highest Value 1. High volume, low value pages. 2. Low volume, high value pages.
  • 28. How to Run Smarter A/B Tests, Faster Finding Broken Links
  • 29. How to Run Smarter A/B Tests, Faster Finding Broken Links
  • 30. How to Run Smarter A/B Tests, Faster Auditing Your Funnel The further down the funnel, the smaller the necessary impact.
  • 31. How to Run Smarter A/B Tests, Faster Auditing Your Funnel
  • 32. How to Run Smarter A/B Tests, Faster Auditing Your Funnel
  • 33. How to Run Smarter A/B Tests, Faster Exploring Internal Search
  • 35. How to Run Smarter A/B Tests, Faster Segmentation Is the Secret bit.ly/gasegmentation
  • 36. How to Run Smarter A/B Tests, Faster Click Maps 1. This info is largely available in Google Analytics via “Navigation Summary”. 2. See when people are clicking on non-links.
  • 37. How to Run Smarter A/B Tests, Faster Scroll Maps 1. Better prioritize your content and messaging. 2. Drastic colour changes mean a need for visual cues… likely indicate false bottom.
  • 39. How to Run Smarter A/B Tests, Faster Session Replays 1. Real users with real money. 2. Where do they pause? Where do they give up? 3. Use this to gather more insight on poor performers.
  • 40. How to Run Smarter A/B Tests, Faster Form Analytics 1. Which form fields cause the most error messages? 2. Which do people hesitate to fill? 3. Which do people leave empty, even if required?
  • 41. How to Run Smarter A/B Tests, Faster On-Site Surveys Exit surveys. On-page surveys.
  • 42. How to Run Smarter A/B Tests, Faster On-Site Survey Questions 1. What was the purpose of your visit to our site? 2. Were you able to complete your task? 3. If not, why? 4. Is there anything holding you back? 5. Do you have unanswered questions? Focus on actionable data, on the friction.
  • 43. How to Run Smarter A/B Tests, Faster On-Site Survey Options 1 open-ended question. Y/N and then an explanation post-answer.
  • 44. How to Run Smarter A/B Tests, Faster On-Site Survey Guide bit.ly/onsitesurveys
  • 45. How to Run Smarter A/B Tests, Faster Customer Surveys 1. First time buyers. 2. No previous relationship with you. 3. Try for ~200 responses.
  • 46. How to Run Smarter A/B Tests, Faster Customer Survey Questions 1. Who are you? 2. What are you using this product for? 3. What problem does it solve for you? 4. What do you like about the product most? 5. Did you consider any alternatives? 6. Why did you choose us over the competition? 7. What’s the one thing that nearly stopped you from buying? 8. What was your biggest concern or fear about us?
  • 48. How to Run Smarter A/B Tests, Faster User Testing Tasks 1. A specific task. 2. A broad task. 3. Funnel completion.
  • 51. How to Run Smarter A/B Tests, Faster Constant Ideation 1. Continue conducting conversion research. 2. Involve the entire team. 3. Focus on the entire funnel (acquisition to referral).
  • 52. How to Run Smarter A/B Tests, Faster “If you come to me with an idea and it’s not live in 2 weeks, it’s not because it’s a bad idea. It’s because I have better things to test.” ~Pauline Marol
  • 53. How to Run Smarter A/B Tests, Faster The Prioritization Problem 1. Not objective enough. 2. Not data-driven enough. 3. Not easily customized.
  • 54. How to Run Smarter A/B Tests, Faster
  • 55. How to Run Smarter A/B Tests, Faster Benefits of PXL 1. Makes “potential” more objective. 2. Makes “ease” more objective. 3. Helps create a data-driven culture. 4. Easy to customize.
  • 56. How to Run Smarter A/B Tests, Faster Prioritization Spreadsheet bit.ly/pxlmodel
  • 57. How to Run Smarter A/B Tests, Faster Better Management 1. Review KPIs. 2. Review tests launched (and not launched). 3. Review insights gained. 4. Choose tests for the upcoming week. 5. Discuss resources (or lack thereof).
  • 58. How to Run Smarter A/B Tests, Faster Archive (Or Live to Regret It) 1. Avoid repeating tests by mistake. 2. Easily communicate wins and insights. 3. Improve the quality of future tests. 4. Create an insights > wins culture.
  • 60. How to Run Smarter A/B Tests, Faster High Velocity Mistakes 1. Without buy-in and a testing culture, you’re doomed. 2. Calling it too soon and non-representative samples. 3. Letting quality and effectiveness slip.
  • 61. How to Run Smarter A/B Tests, Faster Testing Capacity (52 / Avg. Req. Test Duration) x Number of Testable Pages (52 / 2) x 10 = 260
  • 62. How to Run Smarter A/B Tests, Faster Measuring Velocity 1. How well are you trending towards achieving your capacity? 2. Trends are what matters… are you getting closer? 3. Chart it.
  • 64. How to Run Smarter A/B Tests, Faster How to Run Smarter A/B Tests, Faster @shanelle_mullin 1. Conduct proper conversion research. 2. Come up with ideas constantly. 3. Prioritize with data. 4. Manage the process. 5. Archive your learnings. 6. Achieve your testing capacity. 7. Stop listening to CRO BS you read online.
  • 65. How to Run Smarter A/B Tests, Faster Resources Optimization Training Program bit.ly/cxlinstitute Original UX Research bit.ly/cxlresearch
  • 66. How to Run Smarter A/B Tests, Faster Questions? Ask any time via email: [email protected]