A B Testing
In today’s omnichannel battle for consumer attention, it is more important than ever to provide users with personalized offers and online experiences.
To achieve this, many brands—large and small—are developing robust A/B testing practices to understand users better and deliver online experiences that increase conversion opportunities. This can be easier said than done, however. A/B testing is a common practice, and, all too often, it can be tempting just to throw money or technology at the problem. But, few brands find impactful success in their attempts to execute A/B testing to its full potential.
At eClerx, we have significantly built out our skills across A/B testing and Conversion Rate Optimization (CRO) over the past 10 years and partnered with Fortune 100 brands to develop and execute award-winning consumer experiences. While many firms can make these claims, very few can deliver on what matters most—money in the bank. That is where we are different. In Q4 of 2021, we were able to deliver a net sales impact of £111M for a leading global sportswear brand and a $2M revenue uplift for another hi-tech manufacturer. From banking sign-up forms to eCommerce search bar placement and everything in-between, our team of A/B testing experts has seen (and tested) it all.
Missing something that you like? Get in touch to let us know.
Given our success and continuous drive to deliver results for our clients, we have gathered some tips, tricks, and insights over the years to help your A/B testing team get the most value for money out of its tools, people, and processes.
Below, we outline 5 tips that can help streamline processes and optimize your customer experience.
1. Select a tool that meets your team’s needs, maturity, and overarching business goals.
While there is a range of brilliant platforms available on the market, it is important to remember that they all have their pros and cons. When selecting a tool, we recommend starting with the business case and KPIs that you are looking to test. Before going into platform selection, be sure to understand the features you will require, such as filtering or dynamic URLs, heat maps, and integration with your tracking platform. In another piece, we will cover a full list of questions to consider when assessing platforms or getting ready for an RFP, be sure to sign-up to our mailing list to get notified once published.
There is nothing worse than seeing clients spend tens of thousands of dollars on a platform only to use 5% of its functionality. Each tool requires varying levels of technical ability, budget, and resources to deliver the most value, so make sure to agree internally on what you want and speak with industry experts to understand what is right for your team in its current phase of growth. Our team has a wide range of platform knowledge, including expertise in the following platforms.
2. Tools will not solve it all—you need industry experts too. So, do not be afraid to augment your team with external resources.
While investing in technology to solve a problem may seem attractive, it only solves part of your problem. You also need to invest in the right expertise to help execute every stage of your strategy. This includes data analysts to review existing data, strategists to help ideate and prioritize tickets, and—the most difficult but arguably the most critical—platform developers to set up tests and ensure test quality.
These roles are just the tip of the iceberg when it comes to high-volume testing teams, but do not panic if your team lacks these resources. Many of the world’s leading brands outsource these roles, and so should you.
Our team of customer experience optimization experts can seamlessly integrate with your team, filling specialist resource gaps or even handling the entire end-to-end process. Check out our modular framework showcasing how we help Fortune 100 brands win the buyer journey:
3. Good tests are reliant on good data.
This may seem obvious, but the details matter. So, make sure to invest in getting your data right. This begins with a thorough analysis of your existing data to identify testing opportunities and multiple QA checks to ensure tests are correctly set up so that results are accurate.
Also, it is important to invest in a tracking tool alongside your testing platform. As mentioned earlier, the tracking tool should integrate seamlessly with your testing platform to provide insights such as user heat maps and on-page customer experience metrics.
Finally, focus on getting the basics right, such as setting a minimum test length of at least 7 days (ideally 14 days to account for anomalies) and having enough traffic to achieve statistical significance. There are a variety of free tools that can help you calculate this data; check out some of our favorites here:
4. Agree on a standardized process for evaluating, prioritizing, and reporting on tests.
It is easy, especially in large organizations, to freestyle prioritization, tracking, and reporting. While it might appease upper management to test every one of their ideas, this is a recipe for disaster. This is why we suggest having a structured framework for test prioritization. With our clients, we typically adopt the PIE Framework alongside their organizational priorities to ensure that we do not get lost in the ‘test everything’ mindset. Here is an overview of the PIE Framework below:
Potential: How much improvement can be made on this page?
Impact: How important and valuable is this page’s traffic?
Effort: How easy is it to set up a test?
Deciding on what to test is only one-half of the battle; reporting post-test results is equally important. Not only does solid internal reporting ensure you optimize the customer experience and identify future test tickets, it also acts as a reminder to management as to why they should invest in your optimization program. We often see teams attempt this, but a lack of standardized dashboards and reporting processes can slow your testing lifecycle down dramatically. Luckily, our team is well-equipped to help. Our analysts have access to hundreds of reporting frameworks and processes that helped one client reduce reporting time by 40%, increasing the number of tests executed in a quarter.
5. Rome was not built in a day, and neither is a perfectly optimized customer journey.
It can be tempting to finish reading this article and march into your next team call with a plan to overhaul your testing practice entirely, but be patient. The best customer experience optimization practices are built over many months of continuous testing and improvements. If you are just starting to update your approach to testing, we recommend that you do not start too broad. Rather, select a limited set of KPIs you want to improve that align with your broader business goals. Once you have these, start coming up with some test ideas to uplift those key metrics, keeping in mind our PIE framework. Finally, do not be afraid to ask for help—the best apps and websites out there are all relying on tens or hundreds of internal and external experts to execute their test and learn programs.
So, now that you have insights into some of the best practices for getting the most value for your money out of your testing programs, it is time to consider how eClerx can support your team.
With hundreds of strategists, data scientists, developers, and over 50 certified adobe practitioners, eClerx Digital is the partner of choice for the world’s leading brands. Even if you are not in the market today for a new platform or additional expert resources, get in touch with our team to find out how we can support you in future projects or to learn more about our 3-month no-risk pilot.
So, what are you waiting for?
About Authors
Dylan Wingrove
Product Marketing Manager
