How to Increase Your Testing Success by Combining Qualitative and Quantitative Data
3 likes1,189 views
The document discusses strategies to enhance A/B testing success by integrating qualitative and quantitative data. It outlines five key tactics: understanding conversion barriers, utilizing customer language, prioritizing hypotheses, conducting small experiments, and maintaining continuous testing. The focus is on learning from experiments to improve conversion rates effectively.
How to Increase Your Testing Success by Combining Qualitative and Quantitative Data
- 1. How to Increase Your Tes0ng Success by Combining Qualita0ve and Qualita0ve Data Hiten Shah, KISSmetrics
- 2. Hiten Shah Co-‐Founder KISSmetrics @hnhshah
- 3. Hiten Shah Co-‐Founder KISSmetrics @hnhshah
- 4. There’s a big problem with A/B tes5ng!
- 5. You’ve tested the color of your bu?ons
- 6. You’ve watched your compe5tors
- 7. You’ve tried tes5ng everyone’s opinion
- 8. You’ve hit the limits of your current design
- 9. You’ve run experiments inspired by blog posts
- 10. But…
- 11. Only 1 out of 5 tests win. 4 out of 5 tests cost you money. $
- 12. There are no magic bullets or pixie dust
- 13. You need to double your win rate.
- 14. Tools
- 20. 5 tac5cal 5ps to double your win rate
- 21. 1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
- 22. Discover what informa5on is missing
- 23. Ask people when they try to leave
- 24. Email a survey
- 25. Ask people why they cancelled
- 26. Get feedback right when people cancel
- 27. 1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
- 28. Learn what frustrates them
- 29. Find out how they describe your product
- 30. Test very specific call to ac5ons (CTAs)
- 31. 1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
- 32. “A solid test hypothesis is an informed solution to a real problem – not an arbitrary guess. The more research and data you have to base your hypothesis on, the be er it will be.” ! — Michael Aagaard, h p://kiss.ly/abthypo
- 33. Answer key ques5ons for each experiment 1. Which top metric are we trying to improve? 2. What can you anchor your metric against? 3. What is your hypothesis about why this metric is at x%? 4. What do we expect from this experiment? 5. What was the result and what did we learn from it? hEp://kiss.ly/6qsab
- 35. 1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
- 36. Waste less code ! Small experiments help you focus on learning instead of wri0ng code.
- 37. Know what works ! Small experiments help you isolate and understand what impacts conversions.
- 38. Validate your ideas ! Small experiments help you validate assump0ons about your product.
- 39. 1. Find out why people don’t convert 2. Use the exact words customers do 3. Start with a hypothesis then priori5ze 4. Create experiments that are small 5. Con5nuously experiment
- 40. Con5nuous Experimenta5on Small, measurable changes. Keeps us honest. Prevents us from breaking things. hEp://kiss.ly/etsycont Dan McKinley @mcfunley @etsy
- 41. One more thing... Learn more… hEp://grow.kissmetrics.com/op0con NEW! KISSmetrics Customizable A/B Tes5ng Report
- 42. THANK YOU Hiten Shah @hnshah
- 43. Hiten Shah Co-‐Founder KISSmetrics @hnhshah