"The SplitGen Process: The Strategical Approach To Split Testing"

If You Start Split Testing By Making A List of Potential Split Tests Then You'll Never Beat The SplitGen Process...

Old school: We recommend you do NOT start your split tests with deciding what to test.

We decided that we'd write an explanation of how to run split tests that will blow those old results out of the water...

Contrary to popular belief, every split test is not a great split test.

The first thing that many people do when running a split test is decide what they’re going to test. This is a common ‘tactical’ approach that starts with specifics and which can produce wins, no doubt. But it can also cause you to accidentally split test useless factors that have no effect on overall conversion rates.

In fact, previous experiments have taught us that more than 71% of self created ‘assumption’ designed split tests will not increase conversions, and may even reduce conversions.

With such a low margin for randomized success, the overall result of ‘assumption’ split tests are often small minor improvements, or flaky results that leave you scratching your head.

So there are huge variations in the quality of split tests that are possible.

The SplitGen Process

Below is a ‘time tested’ split testing system which we coined the ‘SplitGen Process’. It is a strategical method to develop and generate a list of potential split tests that can produce big wins instead of incremental improvements.

You can adopt this process if you want to turn ‘assumption’ style results into more solid, reliable wins. For best results, compare (or split test!) the following ‘SplitGen Process’ against any tactical ‘assumption’ style of split testing.

The result will be that the following SplitGen Process produces more big wins consistently, time and time again.

The Tactical ‘Assumption’ Approach

From experience, we see most split testing done with a very tactical approach. A tactic is something like changing a buy now button to orange in colour, or adding a ‘satisfaction guaranteed’ logo. Those are tactics.

A tactical approach means the team/person in charge of running the split test will start out with a very rigid idea of what will be split tested and what exact tactics will be used.

They’ll start out with a team brainstorming session or even worse, sit alone and predict what factors will affect conversion rates. Then they’ll use these predictions and self deductions to build a list of things to test. Or they’ll approach the split test with their own bag of tricks, and try to paint every split test with the same brush.

This is the equivalent of taking a questionable recipe and running a single person taste test. Then on that single persons approval, taking the recipe and producing it on a mass scale and expecting everyone to like it. All based on the opinion of one person. Even if that single taste tester was a chef, this method of food production is rarely done in the commercial world. This method is all based on one persons opinion and is bad business sense.

Yet, when it comes to split testing, the common practice is to start out with a pre made list of tactical split tests and then apply that list of split tests to a website etc.

This happens due to the Endowment Effect which tells us that we value our own ideas more then the ideas of others. This is all very well, but the second problem with relying on our gut instincts and estimations like this, is that we tend to only seek validation of our own ideas. So a fair analysis of the split test results is never truly realized, and we only find minor improvements.

So keep in mind that tactical split testing usually leads to minor improvements in results. But even worse, these minor improvements will be seen as an overall success, and so the weakest method of split testing continues over and over. This leads to achieving small wins at times when major wins could have otherwise been produced.

Don’t start a split test with an already pre conceived idea of what needs split testing. You can do that later. First, start with a strategical overview of how to conduct a ‘big win’ split test.
 

The Strategical Approach

In order to find long term ‘consistent’ big wins, split testing needs to be approached strategically. Strategy is a long term overall picture. It doest contain any rigid details like button color etc.

Skillfully planning your split test formula, developing a methodological approach, researching your factors then executing efficiently is all part of the ‘SplitGen Process’. It’s a strategy for producing big wins.

The reason people focus more on tactics is because they’re much more tangible and let’s face it – they’re very sexy. Compare that to strategy, which takes time and effort.

But strategy always works to produce larger wins. If you put in the work to be as strategical as possible, you’ll produce much bigger wins on a much more consistent basis.

The best way to be strategical is to use a split testing ‘process’ – an actual system that acts as an overall strategy. When you run that system, its job is to deliver the tactical factors you need to split test in order to produce big wins that will make a real difference in boosting response.

The System Below - The ‘SplitGen Process’:

The ‘SplitGen Process’ is actually refined down from the system used by top level companies who split test in the real world, to improve manufacturing and sales processes across multiple industries.

There, it’s possible to be working within an industry that you have no experience in at all, and where a website is not being used to sell anything. There are often no ‘button colours’, headlines or CTA’s to test against each other.

So in such an open and seemingly ‘harsh’ environment, there’s no way that you can approach the split test with a pre conceived bag of tricks, or a pre made list of factors and constraints to test.

Split testing has been used to improve the missile manufacturing process since the 1960’s.

Just how exactly do you walk into a weapons factory and use split testing to make the projectile manufacturing process more efficient, when you know nothing about how missiles are made?

If you’re a tactical split tester, you’ll have no clue. But if you use the strategical method of split testing, you’ll understand how to do this right away.

Legendary statisticians like R. L. Plackett, J. P. Burman, Walter Shewart, W. Edwards Demming and Charles Holland, have been refining this statistical process control into an advanced method of split testing for many years.

We then have to take this advanced method of performing split tests and modify it for online use. When applied to a sales website, this strategical approach will consistently produce much bigger wins then if you were using a tactical approach to split testing alone.

The strategical approach explained below has been adjusted so it can be applied to sales websites. It produces consistent big wins.

Here’s how to use the ‘SplitGen Process’ for a strategical approach to split testing.
 

Phase 1: Diagnose

You don’t go to your doctor and say, “Hey doc, I need a heart transplant, let’s get this done!”. Neither does the doctor look at you as soon as you walk in and tell you what you need. The first thing the doctor does is ask questions and diagnose your condition.

Approach split tests in the same way. You shouldn't be brainstorming any split testing ideas at this time.

Don't start your split testing with a single person/team brainstorm of possible split tests!

Ironically, in the manufacturing process, most split tests start with asking the highest paid persons opinion (HiPPO).

However, it was actually found that HiPPOs and top level executives are just as bad as other employees at predicting what factors would increase conversions at a later time. Successful suggestions aren’t limited to any specific level of employee. History has proven that anyone can suggest a winning factor.

It's the same with websites.

You need to see how people use your website. What slows them down? What gets in their way? What stops them from making a purchase? Are there any technical issues that are coming up? Does your site look the same way across multiple browsers? Is there a coding error on your website? Is something confusing your users?

You need to know exactly where the hold-ups are.

If you're not able to watch a group of people using your website, a service like Usertesting.com can be useful. Another great tool for seeing how visitors interact with your website is Crazyegg.
 

Phase 2: Quantify

You need to make sure you understand exactly how you define successes. Are you trying to increase the number of e-mail opt-ins? Are you trying to increase the conversion rate of people who move from page four to page five? Are you trying to lower the cost of each new customer? Are you trying to maximize your earnings per visitor?

Whatever you're trying to achieve, you need to know what specifically it is and how you will measure it. You also need to make sure that your method of measuring these objectives is extremely accurate.
 

Phase 3: The Sales Funnel

Create a visual representation of entire sales funnel. You should seek to understand three important things here:

1. What happens to your visitor before they arrive on your website?
2. What process does the visitor go through whilst buying on your website?
3. What happens to the visitor after they have bought on your website?

Don't create this as a text file. It's very easy to skip this part but this is not the time to be lazy.
You need to have a visual representation of every single step named above. Think about the individual actions that your visitor takes, and draw them out as a path.

This will give you a great overview of the business.

You should understand how a visitor first learned of your website. Did they see an ad or were they a referral? What did the ad promise them? Was that promise kept when they arrived on your website? Where was the ad displayed? What is a visitor from that traffic source like? For example, Google users tend to be more technically minded compared to Bing users etc.

Also, think about the keywords that bring people to your site. What sort of visitors are they attracting? What do your visitors want to do on your site? Is the majority of traffic on your website made of first time or repeat visitors? Is there a specific traffic source that converts very well. For example, you can build a custom landing page for visitors from a specific place etc.

This information will come in handy later on as well.

So try to get as much information about your exact sales funnel as possible, and build a visual representation of it.

The aim here is to look for potential areas of your business where you can make simple changes that will potentially produce big wins.
 

The KISSmetrics Funnel and people reports are useful for locating areas of your business where you can make big gains quickly.

You should also be looking for areas where you can increase revenue. For example, if your website sells shoes, you may be able to replace your current thank you page with an additional purchase offer for a matching belt etc. Or if you sell luxury car wax, you can ask the person if they want to be sent a reminder to buy more wax in 60 days time (when their bottle starts to run low) etc.

Phase 4: Competitive Intelligence

Take a look at your competitors and their businesses. What do they do successfully? Have they been running a specific ad for a long period of time? If so, try and go through their sales process via that ad. If the ad has been appearing for a long time, you can bet it is producing good sales for your competitor. Try and figure out how and why.

You also need to understand what's being said about you and your competitors online. What do people like? Why do they do business with one company instead of another? You need to know this information and use it in your business too.

It's just as important to look at the complaints online too.

What don't people like about your competitors? What don’t they like about you? You should eliminate all of these complaints from your company. This is also a great place to find ideas for improvement and new features to add to your service.

For example, if consumers are complaining that no luxury hair salon offers head massages, you can look to include head massages with your service at an additional price. This will help you drive up overall profitability whilst serving a market need that you didn't even know existed.

Phase 5: Discovering Buyer Friction & Skepticism

A good copywriter always does their research. One of the key pieces of information to understand is what friction a buyer faces and what stops them from buying? What is the buyer skeptical about?

These issues are then acknowledged in the sales copy to remove buyer friction and overcome any skepticism.

For example, there is a very famous stock and bond ad. Louis Engel understood that common buyer friction was caused by the belief that stocks and bonds is a complicated business.

Great ad: Louis Engel overcame skepticism early in this ad (which went on to build the company Merill Lynch).

He overcame the skepticism right at the beginning of the ad with the text:

‘Some plain talk about a simple business that often sounds complicated’. This ad went on to build the company Merill Lynch.

Although it sounds simple, understanding buyer friction online can be difficult. The key process is that you must obtain significant feedback from your visitors. Then you have to use the information you collect to build your website and sales copy so that it naturally overcomes visitor skepticism.

You can use Qualaroo, SurveyMonkey and Kampyle to get visitor feedback on your website, without over intruding on your visitor experience.
 

Phase 6: Shining As Bright As You Can

There are always hidden ‘proof’ assets within a company that they never show to their visitors. You need to find these and make sure that your visitors know about them.

You're basically showing proof that you are:

• An Authority
• Credible
• Experts in your industry
• Trusted & Respected etc

You can do this by showing publications you’ve appeared in, awards you've won, naming previous well known clients (with their permission of course), showing case studies of your results, using testimonials and reviews etc.

Many people think this is about additional credibility. However, it’s more about the extra believability you can provide.

These facts instantly make the chiropractor look like more of an expert, yet he was failing to tell his patients this information.

These are just a few ways that you can prove that you are a good company to do business with.

Another common mistake is not showcasing other products you sell. For example, one client sold a solution to prevent industrial heating systems from becoming clogged up. They also sold an additional product to remove any current build-up inside the heating system, but hardly any of their clients knew about it.

When they added the ‘build-up remover’ to their sales system and told buyers about it, the additional sales almost doubled their profits on that specific product line.

If you’re going to shine bright and sell everything you can, you need to make sure your customers are fully educated about who you are and what you sell.

Phase 7: Squeaks Vs. Roars

At this stage in the system, you should have a whole lot of ideas which were generated from the previous research steps.

But before we design our new webpages, we need to be careful of one last thing. To produce the biggest and fastest wins, some people focus on the squeaks. But we should be focusing on the roars.

The Squeaks

A squeak is an element on a webpage that can only boost conversion rates by small amounts. So for example, changing the background color of your website will only produce small incremental improvements.

We don’t want minor improvements – we want big wins.

The great Gary Halbert once noted that the addition of extra order forms in his direct mail packages boosted response. Then adding another order form also slightly lifted response again and so on. But these are minor wins – they’re squeaks.

If you have a 1% response and are looking to break even on your front end with a 2% response, adding in additional order forms won’t produce the big win you need (besides, this has been tested by us and has never seemed to increase results).

So don’t aim for minor wins -stay away from the squeaks.

The Roars

A roar is the opposite of a squeak. A roar is a specific factor on a webpage that has the potential to boost conversion rates by 100% or more.

For example, switching to a higher quality traffic source, drastically changing the headline of a page, price testing, adding upsells/continuity offers, premium bonus gifts, switching the offer to a free trial etc.

These are the roars that can make big differences to your response rates.
 

ROARS: The effects of price changes can be drastic. These are know as 'ROARS'.

For example, an educational item was being sold for $139. We introduced a split test where the item was sold at $139 as the control, $159 and $169.

The response on the $139 and $159 prices was exactly the same. So the new $159 sale price allowed us to make an extra $20 (14%) on every sale.

But the $169 price cut response in half. So for adding an extra $10 to the $159 sale price meant that we lost 50% of all orders, which meant a lot less profit overall.

So it's important to make sure that any changes you’re making contain as many roars and ‘big picture’ changes as possible.

This way, you'll be running real split tests that mean something. You will see faster improvements much more quickly and most of your tests will reach statistical significance too, which is much more rewarding for you.

Another thing to remember is to focus on simple, fast and cost free changes to make. So don’t switch to an entirely new CMS which will take a long time to implement and cost a lot of money. Start with something simpler.

Look to your previous list of research as well. For example, your research may uncover that a certain need was not being met by your current products. You can then add an existing product that meets that need into the sales funnel and probably see drastic increases in conversion.

Phase 8: Think Outside Of The Box

When running your split tests, try hard to focus outside of the box of what ideas first come to your mind. Definitely look at non-page factors you can play with. Phase 3 will help you with this too.

As an example, sales response was once boosted by 50%+ without making any changes to the actual sales letter. This is a difference between 100 sales and more than 150 sales. That is huge, considering that the exact same sales copy was used in both examples.

Thinking like this will put you light years ahead of your competitors. It’s something lots of split testers don’t do and will give you the competitive edge you need when trying to stay ahead of the pack.

In business, these small advantages all add up.

Phase 9: Designing Your New Webpage

At this stage, we've done our research and eliminated all the assumptions. It's now finally time to design the new webpage we’ll be testing.

If you’ve stuck to being methodological so far, the chances of success are high. You also shouldn't have any problems designing the new page at this stage, because the previous research steps will tell you exactly what is needed on the new page.

As you can see, this system is a well refined process. When setting up any split test, don't be afraid to aim high. Try to avoid starting out with a list of split tests to try. Don’t jump in and start tweaking button colours when they may not mean a thing.

Instead, use the process above and it will help you steamroll the competition if you’re ever going head to head with a tactical split tester (which will be more often than not).

As you can imagine it takes a long time to figure this out and get it working like a system. And it doesn't help that most of the improvements require you to do something that is the opposite of what most business owners do to improve their sales.

If you're interested in having us increase your sales with the SplitGen Process then contact us today. Or you can also get in touch and have a 'Lost Profit' Finder: Diagnostic Session