Disclosure: This content is reader-supported, which means if you click on some of our links that we may earn a commission.
A/B testing (also known as split testing or split bucket testing) is a method of comparing two versions of content such as web pages, applications, email marketing, etc. to determine which version performs better. Essentially, it is an experiment where two or more variants of a page are randomly presented to end users to find out which variant has a higher conversion rate based on statistical analysis data.
Running A/B tests can help you identify key issues about your site or application, take the guesswork out of the optimization process, make decisions based on real data, and shift the business conversation from we think to we know. By measuring the impact of modifications on metrics, you can ensure that every update will produce positive results.
How A/B testing works
In A/B testing, you can make changes to a web page or application to create a second version of the same page. This modification can be either a tweak to the headings or buttons or a complete redesign. Then, when the target page is presented to the visitor, half of the traffic shows the original version and the other half shows the modified version.
When visitors are directed to a proto or variant landing page, they generate engagement data for each experience, which you will measure and collect in your dashboard and analyze through your statistics engine. You can then determine if modifying the experience will have a positive impact on conversions.
Why A/B testing
A/B testing allows individuals, groups and companies to collect user experience data and fine-tune pages accordingly. You can build hypothetical models from this to better understand which elements affect user behavior. In other words, your existing solution may prove to be wrong, and A/B testing can tell you which design is more appealing to your customers.
More than just answering a one-time question or resolving a disagreement, A/B testing can be used consistently and to continually improve the page experience. It's best practice to test one modification at a time, which helps determine the impact of each change on visitor behavior, and your landing page will get better and better over time.
By testing the ad copy, marketers can learn which version attracts more clicks. Then by testing subsequent landing pages, they can also learn which layout best converts visitors into customers. If each step is effective in getting new customers, it will ultimately reduce the overall spend on the campaign.
Developers and designers can also benefit from A/B testing. As long as your goals are clear, then product release, user experience and feature design can be optimized through A/B testing.
A/B test execution steps
You can start running A/B tests by following these steps.
- Collecting data: after analyzing raw traffic on your own, you can then drill down into positions worth optimizing, generally starting with high-traffic areas of the site, as this allows you to collect data more quickly.
- Define the objectives: in order to determine whether the variant is more successful than the original, you should define some reference indicators, such as button clicks, number of registrations, amount of sales, etc..
- Generating hypotheses: Once the objectives have been defined, you can then list ideas and hypotheses for A/B testing and then prioritize them based on expected impact and difficulty of implementation.
- Creating variants: using A/B testing software (e.g. Optimizely) to make changes to elements of the site, perhaps the color of buttons, the order in which elements are arranged, the hiding of navigation bars, or completely customized content.
- Run the experiment: Launch the experiment and wait for visitors to participate! This time, your visitors will be randomly assigned to different landing pages and their experience data will be measured, calculated and compared to determine the final performance of the page.
- Analyze the results: After the experiment is completed, the A/B test software will display the differences between the two (or more) versions, and you can analyze the results accordingly.
- If the variant produces better results, congratulations! See if you can apply the knowledge from the experiment on other pages and continue iterating to generate better pages. If you don't see the expected results, don't get discouraged, summarize the lessons learned from the experiment, and start the next round of A/B testing.
Some suggestions for A/B testing
Media companies
Media companies may want to increase readership, time spent on the site, social shares, etc. It may be effective to test the following variables.
- Email Registration Mode
- Recommended content for readers
- Social sharing buttons
- Travel Company
- Travel companies may wish to increase metrics such as number of bookings, ancillary purchase revenue, etc. The following variables can be tested.
- Home Search Mode
- Search results page
- Auxiliary products introduction
- E-commerce companies
- E-commerce companies may want to increase the number of completed orders, average order value, holiday sales, etc. The following variables can be tested.
- Home Promotions
- Navigation menu
- Checkout page (6 steps to optimize the checkout page)
- Technology companies
- A technology company may want to increase the number of quality leads, the number of free trial users, specific types of buyers, etc. The following variables can be tested.
- Form Components
- Free Trial Process
- Home page messages and call-to-action phrases