A/B Testing summary

Are you ready to start optimizing your website? Well, GREAT. But before you begin, you need to consider exactly what you want to test. Because without a definition of success and a hypothesis, the test won’t tell you much.

So start by clearly defining quantifiable success metrics – that is, metrics which measure whatever data is most relevant to you – in order to evaluate the testing later.

Although A/B testing is great for tweaking and refining your online presence, it can also lead you to make major, widespread changes in how you structure your site.

This was what Disney experienced when it performed an A/B testing experiment on the homepage of one of their TV networks, ABC Family. When they examined their search logs, the Disney team noticed that a lot of visitors were searching for specific shows. So instead of making minor tweaks, they decided to create a completely different homepage structure which listed all the shows, making them easier to find.

Imagine visiting a website and being immediately bombarded with images, pop-ups and requests to sign up. No doubt you would leave as soon as possible and never return.

This is something most of us intuitively grasp: When it comes to site design, less really is more. In fact, A/B testing has shown that removing any fields that aren’t absolutely crucial will have a huge impact on engagement.

An image says more than a thousand words, as the saying goes. And although there’s a lot of truth to that, A/B testing shows us that if you really want to engage people, you need to find the right words.

And one way language can help make your website more effective is by making the meaning of clicks clearer to the end user.

Scientists often say that there’s nothing more exciting than discovering something that doesn’t work, because you can learn so much from failure. Well, exactly the same applies to A/B testing: Failure can be a blessing.

This principle applies to every sort of failure, including the small ones. Because even dead ends that seem insignificant now can be useful later on.

Just as political movements need activists and lobbyists to bring about change, A/B testing needs evangelists. And if you want to implement this practice within your own organization, you might need to become a spokesperson for it.

And one of the best ways to convince other people of the impact of data-driven approaches is to communicate the results of simple tests that display the immense value of A/B testing. Simply presenting images of variations and test results on a regular basis will convince stakeholders that data-driven testing works, thus bringing enormous benefits.