A/B Testing in Referral Marketing: Top 5 Mistakes to Avoid
At Mention Me, we've run over 45,000 A/B tests across ten years, so we know what works. We also know where most brands fall short.
In the words of our CEO, Tim Boughton: A/B testing is essential in setting up a successful refer-a-friend channel. Without the insights and flexibility A/B testing brings, most brands are lucky to reach 3-5% of new customer acquisition from a referral programme. With A/B testing, we routinely reach 10-30%. It makes a huge difference.
So, what are the five biggest A/B testing mistakes to avoid? Read on to find out.
Mistake 1: Not giving your experiment enough time
We recommend running your tests for at least two to three weeks. First, you’ll need to keep in mind your traffic and order volume. A higher volume of orders means your A/B tests should reach statistical significance more quickly.
The length of your purchase cycle matters too. A short purchase cycle and high likelihood of impulse purchase mean new customers who have been referred shop at your site relatively quickly, and then the friends who sent them also buy again soon.
This means that you can understand the impact of refer-a-friend marketing on the full extent of your funnel more quickly.
For many businesses, the combination of traffic and order volume with their purchase cycle dynamics can mean letting some tests run for a month or more.
This might come as a surprise for internet marketers who are accustomed to running simple, short-duration A/B tests on single-page traffic that reach significance in as little as a few days.
Mention Me's A/B testing platform shows you everything you need to know in one place
Mistake 2: Stopping after the first test
This is a mistake we hear time and time again: “We tried an offer, but it didn’t work, so referral isn’t appealing to our customers." In our experience, most clients who launch a referral campaign with Mention Me do not start with the right offer.
Our COO, Simon Dring, says: “For no client in Mention Me history has the first referral offer been the best offer. In fact, it's quite the opposite — all Mention Me clients have drawn significant benefits from a programme of testing.”
Our clients that continuously optimise A/B tests typically see a 4x boost in customer acquisition in just six months.
Most clients find the best results come from a programme of testing. Often the second test yields a huge jump in performance. So, don’t give up at that first hurdle!
Mistake 3: Holding back from testing discounts
We have many, many clients (especially in the luxury end of the spectrum) who feel very strongly about avoiding discounts. If this is true at your brand, then you're in very good company.
Deciding not to test discounts doesn’t mean no testing.
There are also a variety of non-discount incentives you can test (e.g., free gifts or delivery). But testing is testing, and it’s worth being open-minded to what might be most effective.
The nice thing about A/B testing with Mention Me is you could just dip your toe in the water and test a small discount to a subset of your customer base.
With A/B testing by cohort, you limit discount exposure to an exclusive group, so you can decide whether to discount or not based on real data without disrupting the user experience.
It’s also worth pointing out that whilst over-discounting can be very dangerous, there is a big difference between sending customers discounts and offering customers incentives for helping you spread the word. We’ve written about that previously in our blogpost to discount or not to discount.
Mistake 4: Trying to reinvent the wheel
We’ve seen countless clients try to reinvent the wheel when it comes to running A/B tests. There’s no one-size-fits-all, which, ironically, is the beauty of A/B testing. This is precisely where the true benefit of testing shines: tailoring approaches to fit your individual brand needs and audiences.
But why not get inspired by other successful tests instead of starting from scratch each time? For example, take a look at our Vertbaudet case study, where leveraging referral marketing and A/B testing enabled them to win customers who return twice as often.
And check out these proven A/B tests from leading brands to optimise your referral marketing metrics.
Paula's Choice leveraged data to test 'give' vs 'share'
Mistake 5: Making decisions with only partial data
Like all online processes, refer-a-friend can be visualised as a funnel. It's complicated because the full funnel spans your original customer, their referral behaviour, their friend, and their friend’s purchase, and finally back to your customer’s redemption of their referrer offer (yikes!).
This process starts in your shopping cart but can take you far and wide: email, mobile, social media, even offline word of mouth down at the pub can be stopping points on the journey!
Why is this important? Because in order to make a good test decision, you need to consider the impact at all the steps of the funnel as well as throughout the full process.
Sometimes, a test might reduce the rate at which customers share offers (which happens early in the funnel), but it could lead to greater success in later stages, such as when the referred friend makes a purchase. In such cases, the overall positive outcome outweighs the initial decrease in sharing.
Key tip: Test, test, and test again
Launching your refer-a-friend campaign is only the start. By avoiding these five mistakes and creating your own series of experiments, it’s possible to see a four-or-five-fold improvement in the performance of your referral programme over time.
It takes a little time and effort to reach its full potential, but getting referrals working well should yield a channel that boosts acquisition by 10% to 25%.
A/B testing doesn’t have to be complicated as long as you keep the consistency of experience for each test cohort. Testing by cohort is critical to unlock referral. Look for a referral platform that enables you to do this so that you can be sure that your customers who have been shown an offer will continue to be able to share that offer even if you want to test something else.
So, what should you look for when setting up a referral programme to make sure you’ll be able to test and optimise?
- Testing by Cohort - test different offers against each other whilst allowing customers to receive a consistent experience.
- Segmentation – serve different offers to different segments.
- Easy Configuration of Offers – change and test variables (e.g., incentives, share options) without development support.
- Detailed Reporting – data at each step of the referral funnel.
- Content Management System – amend any customer-facing text.
- Easily Editable Designs – the design of the referral programme should fit your brand and be easily editable.
Take these tips, avoid these mistakes, and you should have everything you need to run an A/B test campaign that drives real results. Good luck!
Never miss another update
Subscribe to our blog and get monthly emails packed full of the latest marketing trends and tips