Conversion Whitepaper

Conversion Architecture:

Improve Your Online Performance

pr ou an

While Reducing Your Spend

WS
SI White Pa
aper

Pre
epared by:
: Doug Schhust
Con
nversion A
Architectur
re Expert, W
WSI

Improve Your Online Performance While Reducing Your Marketing Spend

Introduction
Research has shown a major difference in conversion and return on investment between companies
that test their online communications and those who don’t.

So why doesn’t every online business have a regular testing program?

The fact is that many businesses and marketers are confused about where to begin, and how. Some that
do test don’t do it often enough, or are not sure how to interpret the results.

With so many potential areas to test, including home page, landing pages, pay‐per‐click ads, emails and
now the influence of social media, marketers need to know:

• Which tests provide the biggest potential return on investment

• How to structure tests for consistency and accuracy

• How to continue improving test results and duplicate success

First of all you need a solid benchmarking framework for basic online testing that enables you to realize
significant gains in conversion and revenue.

We hope this overview helps your business and if you have not yet implemented a formal program as
well as those looking for higher conversions, we recommend a WSI Benchmarking Program to guide and
improve your efforts. Please contact a WSI Marketing Consultant today!

There is only one key thought process you need to act on to increase your online conversion rate and
that is simply Test, Test and Test.

Within that testing there are 4 key areas you need to understand to increase your online conversion and
reduce your overall cost per lead.

‐ Ask The Right Questions
‐ Measure The Right Items
‐ Perform The Right Tests
‐ Analyze & Make Adjustments Based On Results

These tests will give you a better understanding of: your customer profiles, their online behavior
patterns and what drives them to take action.

Copyright © 2009 by Research and Management. All rights reserved. Page 2 of 12


It All Begins With Testing To Establish Benchmarks:

At WSI, we apply the concept of “benchmarking” to our testing process.

What do we mean by benchmarking? A testing process that is useful, practical, and delivers predicable
results that you can carry forward into a successful marketing program.

The benchmark results of our online testing process are directly related to five key elements.

This formula was created by MarketingExperiments and is the basis for online conversion today.

Wherein:
u = Utility
q = Research Question
t = Treatment
m = Metric System
v = Validity Factor
i = Interpretation

In other words, ask the right question, measure the right thing, perform the right test, and learn from
the results.

We recommend that businesses interested in establishing a strong online presence use the WSI
Benchmarking Program and incorporate these elements into your online strategies.

There are 3 main areas that need to be tested, home page, landing page, email and the impact of social
media. Using these testing techniques you can see what influences a client or potential client when they
connect with you online.



THE BENCHMARKING PROGRAM

1. Start with the right question.
In a recent Web clinic on this subject, more than 60% of the audience said that the very first
step in successful online testing—asking “the right research question”—is their biggest
challenge.

All single factor A/B split tests begin with “which?”

Marketers may start with the interrogative “what?” or “why?” when they are brainstorming on
an informal basis, but their research question must progress to “which?” in order for it to be
specific enough to be used in a formal test.

For example, “What is the best headline?” becomes “Which headline will convert best?” and
“What is the best price?” becomes “Which of three price points is best for this product?”

2. Develop the Treatment(s).
A variable is the general element you intend to test. Headline, price, and copy are all example of
variables, and there are dozens more. Again, one key to successful A/B testing is knowing which
one you want to single out.

A value is the specific option (content, size/amount, color, position) of the variable you are
going to test.

The test Control—“A”—will be your current page, ad, or email.

The test Treatment or Treatments—“B” or “C”— are the pages, ads, or emails containing the
new values of your variable.

Values become Treatments. When a new Treatment performs better (better than the Control,
better than another Treatment if there is more than one), it becomes the new Control in
subsequent testing.

3. Determine the right metrics.
Your main objective will be to determine the metrics that will best answer your primary
research question (the “which?”), though secondary metrics may also provide insight.

First, establish what you are trying to accomplish.



Are you trying to attract more subscribers?

Capture more qualified leads?

Obtain a higher conversion rate?

Produce a better ROI on a pay–per–click (PPC) campaign?

After establishing what you want to achieve, choose the element you wish to test in order to
select the appropriate metric.

There are essentially four elements you can measure:

• The amount of activity on your site

• The source of that activity

• The nature of that activity

• The results of that activity

Use organizing questions to help you choose the appropriate metric; for example:

Question Metric

Who visited my page?   Number of unique visitors (UV)

Where did they come from? Referring URL, organic search, PPC ad & social media influence

What did they do/view?   Page views per visitor; clicks

What did they buy / sign up for
Number of orders, average amount, total revenue
or did they call?

4. Check validity, starting with sample size sufficiency.
For confidence in your findings, test results must be statistically meaningful. In other words,
there has to have been enough difference in results to make the meaningful right choice.

Typically, researchers will tolerate no more than a 5% chance that the test results are due to
random variation. This is referred to as “5% significance level” or “95% confidence level.” For
example, you would want to be at least 95% confident that the results of your test will detect a
difference in clickthrough rate of 0.5% or more.



Key point: as sample size (n) increases, Margin of Error (E) declines.

Check for four additional, major validity threats.

There are a number of threats to the validity of testing, but these four threats are common in
online testing:

• History effects

• Instrumentation effects

• Selection effects

• Sampling distortion effects

Ensure that your test results have not been skewed or invalidated by these threats.

History effects. The effect on a test variable by an extraneous variable associated with the
passage of time. Industry events, news events, and holidays are all examples of history effects
that could skew your test results.

Instrumentation effects. The effect on a test variable by an extraneous variable associated with
a change in the measuring instrument. Having a server go down in the middle of your test is one
example of an instrumentation effect.

When reviewing your test protocol to exclude this threat, ask: Did anything happen to the
technical environment, measurement tools, or instrumentation that could have significantly
influenced the results?

Selection effects. The effect on a test variable by an extraneous variable associated with
different types of subjects not being evenly distributed between experimental treatments.

Mixing traffic sources can skew results. A great landing page created for one specific channel—
for example, a PPC ad with great copy related to a specific organic search term—can do poorly
when presented to visitors coming from a different PPC ad for a completely different product or
service.

Sampling distortion effects. The effect on the test outcome caused by failing to collect a
sufficient number of observations.

Depending on the amount of traffic your landing page normally gets, the number of email
addresses you have to send your new email to, or the clicks you normally get on your existing
PPC ad, it may take days, weeks or even months to reach the number of observations necessary
to achieve testing validity.



5. Correctly interpret the results.
If results are valid, determine the impact and decide what your next test should be. Remember,
the “winning” Treatment becomes the Control for your next test.

If results are invalid or inconclusive, consider re‐running the test, but keep in mind that even
invalid or inconclusive tests can provide value when properly analyzed. Learn from your results:

• Expertise is cumulative knowledge. Keep careful records of tests and results. When results are
surprising, look for hidden sources of error or insight.

• Lack of difference is meaningful. That changing the variable(s) in the way that you did had little
effect on performance is valuable knowledge. This may be the single most valuable insight you
gain from an inconclusive test.

• Try stratification. Try filtering the results, looking for patterns (days of the week, hours of the
day, etc.). You may find a clear statistical “winner.”

• Note alternative or secondary conclusions. Look for patterns of performance, even those that
identify things that you definitely DON’T want to do.

Breaking Down Your Campaigns and Marketing Channels
Helps to Determine the Following:
Who your clients are, what their needs and wants are and what drives them to take action. These are
the key to any online marketing program. The diagram below illustrates a standard conversion funnel.



Testing allows you to identify your clients or potential client’s profiles which ultimately leads to higher
conversions. Understanding each potential audience allows you to communicate the right message at
the right time.

Understanding how your audience makes their decision is very important. The graphic below illustrates
a standard buying process in a B2B environment buying process.



Test Protocol for A/B Split Testing for Landing Pages:
1. Develop Your Capabilities and Select the Right Tools
A/B split testing tools vary from simple CGI scripts to sophisticated software applications. You
will find a list of services in the Resources section at the end of this report.

Even without sophisticated A/B testing capability, sequential testing offers you an opportunity
to learn many insights about your pages. For more on sequential testing, see the Concluding
Comments, below.

2. Identify Your Established Control Page
Your control page will be the page against which you test all subsequent optimization efforts. If
you are just getting started with A/B testing, your control page will be your current landing page
before any optimization. When a new page performs better than the existing control page, it
then becomes your control page in subsequent testing.

3. Establish Your Testing Goals and Parameters
What are you trying to accomplish with A/B split testing? Are you after more subscribers, a
higher conversion rate, or a greater return on investment on your PPC campaigns? Your goals
will determine your testing parameters, which will determine the potential success of your
testing efforts.

4. Determine Your Sufficient Test Interval
This time period should allow you enough time to gather sufficient data to gauge real insight
about your A/B tests. Identify the number of unique visitors and/or conversions needed to
establish good data and then determine how long it will take you to generate this traffic. This
number will vary from business to business, but should give you enough data to confidently
declare a "winner."

5. Create 1‐3 Radical Redesigns
KEY POINT: These pages are not subtle optimizations changing only one or two elements on the
page, but are wholly different pages representing a radically different approach.

6. Evaluate These Redesigns in A/B Split Tests
Test these alternate landing pages against the control page. Ideally, each page will be tested
against every other page, but if that is impractical, test two pages at a time and keep the best as



your control for subsequent testing. We like to use heat maps to help track the impact of all the
key elements and its potential impact.

7. Based on Results, Determine Your True Control Page
The radical redesign method will be more likely to generate a quantum leap in improved
conversion rate than optimizing a mediocre page with little potential. Once you have identified
the best general approach, you are now ready to optimize individual elements on the page.

8. Optimize with Traditional Variable‐Specific A/B Testing Variables
to Test:
• Headline
• Call to Action
• Page Copy
• Graphics
• Color
• Configuration of Page Elements
• Etc.



Concluding Comments:
There are a few important points to remember about A/B split testing:

1. Even if you can't set up a true A/B split test (where two versions of a page are being displayed
one after another to alternate visitors to your site), it is very easy to create a sequential A/B split
test.

KEY POINT: A sequential test is when you show one version of a page for a certain period, like
two days or a week, and then show another version for the following two days or a week. The
results may be a little less reliable, but can still yield valuable information and trends.

2. Testing gives you the opportunity to maximize conversion rates, solve problems, and challenge
assumptions. And keep in mind that you have opportunities beyond testing small changes to a
page.

You can also challenge an existing page by designing and writing a radically different version,
where almost everything is different. In fact, it is through these dramatically changed
approaches that you are most likely to achieve breakthrough improvements.

3. Testing provides companies with an invaluable means to demonstrate to company heads and
management the hard figures behind any suggested changes or improvements.

4. Use of consistent testing will increase the knowledge base of your web group or company
significantly. You will learn more, and soon be able to determine a set of optimized practices
that work best for your particular business.

The absence of rigorous testing leaves you in the dark, depending on guesswork alone when
creating your pages.

Conclusion:
Most businesses today do not take advantage of how important it is to test regularly. The key for any
business wanting to achieve online success is knowing how to communicate to your target audience and
knowing that all parties do not act on the same principles. This also allows your to figure out your online
ad spend in an effective manner. The average online conversion today is 2%, which means that 98% of
your ad budget is not being used effectively.

The key is really to keep the testing simple and uncomplicated and once you have established your
benchmarks then you can start to divert from your Control Page. This can be a simple process and or a
more complicated one but the key is simply testing, testing and more testing of the right things and you
will increase your online conversion.



About the Author

Doug Schust’s background consists of both online and traditional marketing. His main
focus is online marketing and helping clients focus not on traffic but the results from
that traffic. Most recently Doug has spoken at several Google events on conversion
and continues to share his knowledge and experience with other consultants and
clients.

Resources:
• MarketingExperiments

• MarketingSherpa

• Omniture

• Google


Conversion Whitepaper

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to Conversion Whitepaper (20)

Recently uploaded (20)

Conversion Whitepaper