Downloaded 30 times
Uploaded byOne to One
Content Testing & Optimization (CTO) Webinar, April 29 2010
This document discusses content testing and optimization (CTO) strategies to increase online conversion rates, revenue, and ROI. It describes different testing formats like A/B/N testing of content elements and multivariate testing of individual page elements. Case studies show how CTO led to significant lifts in metrics like conversion rates. The document recommends establishing objectives, conducting tests in waves, and using learnings to continually improve content and build an optimization library.
More Related Content
![[CXL Live 16] Opening Keynote by Peep Laja](https://cdn.slidesharecdn.com/ss_thumbnails/01peeplaja-openingkeynote-160401004912-thumbnail.jpg?width=640&height=640&fit=bounds)
![[CXL Live 16] Growth Hacking BS: Fixing Marketing One Truth at a Time by Morg...](https://cdn.slidesharecdn.com/ss_thumbnails/xfwvgymotos2fajqo19v-signature-42db88fde2139044dd6963018b134da7e709739587431e34ce8d4285185c8547-poli-160403200010-thumbnail.jpg?width=640&height=640&fit=bounds)
![[CXL Live 16] A/B Testing Pitfalls: Getting Numbers is Easy; Getting Numbers ...](https://cdn.slidesharecdn.com/ss_thumbnails/twc7xa6lqyyg9dmu8wzl-signature-42db88fde2139044dd6963018b134da7e709739587431e34ce8d4285185c8547-poli-160403200010-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Content Testing & Optimization (CTO) Webinar, April 29 2010
![Support, Monitoring, Continuous Improvement & Scaling Agentic Automation [3/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day3-cfd-251120170304-ddef8112-thumbnail.jpg?width=640&height=640&fit=bounds)
Content Testing & Optimization (CTO) Webinar, April 29 2010
- 1. Content Testing &Optimization (CTO)April 29, 2010 One to One Interactive, 2010. All rights reserved.
- 2.
- 3. What Is ContentTesting & Optimization?%?Sale!(But how many made it through?)
- 4. The Main Question“Canwe rapidly test new landing page elements -- or entirely new pages -- to continue to lift our site’s conversion rate?”
- 5. IntroductionWho should considerContent Testing & Optimization?Marketers looking to increase online conversion, revenue and ROI by leveraging the unique opportunities offered by the digital space to improve visitor responseWhat will we cover today?The philosophy, planning and implementation that drives a Content Testing and Optimization (CTO) strategy and share a few of One To One’s successesWhat will be my takeaways?The digital marketing strategy behind Content Testing and Optimization
- 6. How A/B/N andmultivariate test strategies are designed
- 7. How CTO canlead to significant lifts in conversion, revenue and ROI
- 8. How optimization learningscan improve ongoing marketing strategyOne To One’s Approach To Content Testing and Optimization(CTO)
- 9. Approach To ContentTesting and OptimizationObjectivesCreate ROI positive sustainable improvements in content performance against desired actionsBuild ongoing optimization engineApproachConduct test in waves against specific objectives Design tests to always achieve statistical significance with rapid turnaround times (2-6 weeks)Test FormatsContent development (A/B/N testing)Content elements (multivariate optimization testing)
- 10. A/B/N Redesign TestThe“A/B/…N” approach provides the simplest approach to Content Testing and OptimizationRedesign BChampion PageRedesign A
- 11. A/B/N Redesign TestPresenthighly varied, competing page concepts designed to demonstrate as widely different designs as possibleRedesign BChampionRedesign A
- 12. A/B/N Redesign TestThegreater potential to affect overall results temporarily supersedes your ability measure the effect of specific elements on the pageRedesign BChampionRedesign A
- 13. A/B/N Redesign TestDon’tget trapped trying to keep any elements “consistent”. Your visitors will only ever see one version!Redesign BChampionRedesign A
- 14. Most popular complaintto A/B/N testing“But I don’t know why that page won!”Solution?Eye Tracking
- 15. Using Eye TrackingTo Identify Areas of InterestBABABUnderstanding which elements on your winning page generated the most fixation and longest gaze duration can help provide the “why” of A/B/N testingA
- 16. Multivariate Testing (MVT)CompletionOnce an A/B/N test’s winning design is determined, the losing designs are turned off and the winner is declared the new Champion
- 17. Any findings aredetermined via methods such as eye tracking or link analysis, though the specific “what’s most important on this page?” answer may not be available… yet
- 18. At this point“multivariate” or “MVT” testing of specific page elements on the new Champion is recommendedMultivariate Testing (MVT)Elements within a layout such as imagery, copy, calls to action are varied and tested against each other in differing combinations, or “recipes”Multivariate Testing (MVT)The best performing recipe (for example, Headline “1” + Image “3” + Call To Action “2,”) becomes the winner and is rolled out into the next ongoing campaignMultivariate Test DimensionsSample elements for test considerationCalls To ActionImageryHeadlinesContent/MessageQuantity
- 19.
- 20.
- 21.
- 22.
- 23.
- 24.
- 25.
- 26.
- 27.
- 28.
- 29.
- 30.
- 31. Style – lifestyle,product
- 32.
- 33.
- 34. Style – storyv. straight
- 35.
- 36.
- 37.
- 38.
- 39.
- 40. Style – storyv. straight
- 41.
- 42. Banner ad imagematching
- 43. Search / bannermessage or offer matching
- 44.
- 45.
- 46.
- 47. Feature information inpop-ups
- 48. OffersTest Wave ApproachTryto project out ongoing lift prior to launching any test, then benchmark against your projections to ensure a continued positive return on investmentImpacting Success MetricsAction rate after Wave 6: 2.60%MVT OptimizationsRedesign TestsLift: 63%Base action rate before testing: 1.60%
- 49. Accounting For InteractionEffectMaybe surprisingly, the sum of the winning MVT elements is often NOT the best possible pageTest 1Test 3Test 2Test 4Test 5Theoretical Best Recipe#2 HeadlineImage 1
- 50.
- 51.
- 52.
- 53. Secondary CTA 2AccountingFor Interaction EffectTests can be completed more rapidly using the popular “one element at a time” approach… but at the risk of missing better pagesFind Out If Service Is Available In Your Area!Lorem ipsum dolor sit amet, consectetur
- 54. Adipisicingelit, sed doeiusmodtempor
- 55.
- 56. Aliqua. Utenim adminim veniamSign Up To Hear Our Latest Offers!Lorem ipsum dolor sit amet, consectetur
- 57. Adipisicingelit, sed doeiusmodtempor
- 58.
- 59. Aliqua. Utenim adminim veniamRecipe AAll Winning Elements Options From Wave 1Conversion Rate: 16.7%Recipe DAll Winning Element Options exceptHeadline #2Conversion Rate: 23.5%
- 60. OTOi’s Content Testing& Optimization PhilosophyDetermine test objectives, e.g. Clickthrough From Page vs. Action Rate vs. Conversion Rate
- 61. Start with theideal optimization processOptimization Wave: IdealRedesignOptimizeOptimizeRedesignOptimizeOptimizeThe greatest growth in conversion is usually seen with alternating waves of redesign and optimization
- 62. Estimate actionable volumebased on projected traffic
- 63. Extend and/or removetests due to insufficient volume or timeOptimization Wave: Lower-Volume Traffic SourceRedesignRemovedOptimizeRemoved
- 64. Recommended Success MetricsTypicaltesting success metrics include:Conversion rate (share of visitors completing a sale)
- 65.
- 66. Average shopping cartvalueSecondary success metrics include: Bounce Rate
- 67. Action Rate (percentageof visitors completing a valued non-sale action)
- 68. Funnel Depth (averagenumber of steps through the purchase funnel taken)Beware Incomplete FalloutSome tests may be rightfully set up to only measure clicks off the test page
- 69. Be careful thatsuccess isn’t determined by a point too early in the funnelReach Step40%Reach Step34%Reach Step28%Convert18%Landing Page A40% ClickthroughStep 1Step 2Step 3ConversionReach Step30%Reach Step27%Reach Step25%Convert23%Landing Page B30% ClickthroughStep 1Step 2Step 3Conversion
- 70. Forecasting Incremental RevenueConversion RateProjected Conversions1-Mo VisitsProjected Annual Visits=2.00%Winning Design24,000x 12 =x100,0001,200,000=1.80%Champion Design21,600Key QuestionsHow many visits does your tested page receive per month?How long will you continue to award credit to the new, winning page? 12 months?How many conversions or other desired outcomes is your winning page expected to provide over your Champion?What’s the value of each desired outcome -- in terms of average lifetime revenue if available?Projected Additional Annual Conversions+2,400xSample Return EstimateEstimated LifetimeValue Per Conversion$600Projected Total Incremental Revenue$1,440,000 Revenue - Cost= ROICost
- 71.
- 72. Case Studies: Comcast2009 ResultsTest ChampionsTest WinnersConversion Rate Metric11% Lift In Conversion RateWave 124% Lift In Conversion RateWave 266% Lift In Conversion RateWave 336% Lift In Action Rate(over lowest performer)Wave 4
- 73. Case Study: 2009Wave 3 Multivariate TestA 2009 Comcast multivariate study focused on testing:ImageHeadline / Call To Action The test audience was all visitors who had clicked on a Comcast Paid Search ad
- 74. All versions ofthe new page’s design were tested against the prior Champion pageCase Study: Comcast 2009 Wave 3 ResultsImage Results% Lift In Conversion Over Champion“Married Man”“Orange Woman”Headline/Copy Results% Lift In Conversion Over ChampionThe redesign’s template seemed to significantly beat the Champion no matter which combination of Image and Headline/Copy were used
- 75. Among the winningImages, those with the people facing toward the headline and next step button seemed to perform the best (see above)Case Study: Comcast Business Class“Champion” Landing PageTest Hypothesis:“Versioning a banner ad’s landing page to match the design of the banners will increase clicks to the next page”Methodology:Version “Champion” elements against variations designed to match the banner ad creative more closelyElements Matched:Primary Headline Viewed
- 76.
- 77. Call To ActionColors123
- 78. Case Study:Variable Elements“Champion”Elements“Small vs. Big” Elements“Dream Big” Elements“Compare” ElementsHeadlinePrimary ImageCall To Action Colors
- 79. Case Study: WinningPageswinnerwinnerQuestion: “Does matching landing page content to the campaign creative shown make a difference in clickthrough?”Answer: For all creatives tested: yesMixed Landing PagesCreative Clicked OnFully MatchingChampionwinnerwinner
- 80.
- 81. Building Your LibraryFirmsshould view Content Testing & Optimization as an ongoing process which yields an ongoing lift in conversion
- 82. Never forget thatyou’re looking to build out a library of best practices to inform future designsBuilding Your LibraryPast studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:Images perform far more successfully on the right side of the page, potentially due to the natural “inverse C” path of eye trackingBuilding Your LibraryPast studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:For consumer products, images of people are far more successful in driving conversions than images of products – especially of facesBuilding Your LibraryPast studies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:The newer an offering, the less successful hard-sell messaging such as “Sign Up Now!” is to drive page interactionClick To Sign Up Now!Check Out Products And Features!
- 83. Building Your LibraryPaststudies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:Images of happy consumers looking in the direction of Calls To Action rather than out of the screen generally generate higher conversionOffer!Offer!
- 84. Building Your LibraryPaststudies have produce insights which have radically changed the prior approach to content, as ongoing testing has driven substantial lift in conversion via such findings such as:Matching landing page messaging to the copy in a paid search advertisement clicked is far more important when the copy includes an offerIn ConclusionA Content Testing & Optimization strategy effectively leverages the unique opportunities offered by the digital marketplace and provides a foundation of best practices for ongoing content development
- 85. By engaging inthe ongoing process of content testing, firms can rapidly implement new content to generate increased response, resulting in steady lifts in conversion, revenue and return on investment (ROI)
- 86. GIVE IT ATRY!