What Is A/B Testing and Do I Really Need It?

“This single practice could mean the difference between gaining one new client, and gaining a hundred.”

The short answer is yes—A/B testing is absolutely essential if you want to gain more conversions. By calling every element of your website design into question through these tests, you can know exactly what visitors respond to positively, and which parts they do not. As we have stressed in previous articles of this series, the goal is to make informed decisions concerning every detail of your design. You cannot optimize the function of your site by guessing your way through it.

The Test: Revelation Over Intuition

The jargon associated with A/B testing (also called split testing) sound like sports metaphors, which is appropriate because it is basically a competition between the two or more designs to be considered for your final version; there will be 1 winner. It may sound like some complicated operation used by NASA engineers, but the test is actually pretty simple. You take two or more design templates, allow people to view each one, and whichever design receives a better response is the one you use. You can use this test on a consistent basis to ensure that, whenever you add something new to your website, the design is optimized according to the preferences of your visitors, which leads to more conversions. Here’s how it works:

The pages you are comparing against each other are your variants—two or more versions of the same page (included on the page is whatever content you need to test: “start here” landing page, subject line, etc.). If you have run the test before, then the winner from your last test is the “champion”; any new versions included in the current test are “challengers”. Your CTR (Click-Through Rate) is the operative factor here; whichever version gets more clicks on the element being tested is the winner.

Narrow the Scope of Your Test

The most revealing and effective way to run this is to only test one item at a time, wherein the two or more versions of the page in question are identical, except for one variable. For example, if you want to know if a red call-to-action button performs better, or an orange one, then this button should be the only difference between the pages. That way you can be sure that nothing influenced a visitor’s decision to favor one version over the other, which would confuse your results.

Professionals Have Been Doing it for Decades

“If a behavior changes when we vary an experimental factor, then we infer the factor is having an effect.”

Of course, the method behind A/B testing is not a new idea, and in fact, this kind of testing for the purposes of web design were probably born out of the experimentation that professional researchers have been conducting for a long time. In conventional scientific research, there is always careful study and consideration of cause and effect, to the end of discovering exactly what works and why. In their tests, just as in ours, there is typically an experimental group of participants (variant) to whom a variable is applied, and a control group to whom the variable is not applied. Through testing, researchers are able to “isolate the effects of one or more factors by manipulating the factors of interest and holding constant other factors.” As renaissance men and women of our technology-oriented world, we can also trust these practices to be reliable, which is why split testing is so useful in making educated choices in your web design.

So the next time you decide to change the font of your subject line, the color of your landing page background, or the position of your comment section–run an A/B test. This single practice could mean the difference between gaining one new client, and gaining a hundred.

Check out this post on user experience to gain a more complete understanding of intuitive design:
User Experience 101: A Philosophy of Simplicity