Unlocking the Power of AB Testing in Your Referral Marketing Strategy
You know that to maximise your success with referral marketing you need well-designed A/B testing that captures meaningful results.
But your customers are not all the same. They're at different points of the buying cycle, spend different amounts and have different attitudes to your brand.
So how do you control for all these factors? And how can you use A/B testing to produce results you can use to inform your messaging across customer segments?
The most successful brands we work with continually split test elements of their referral programmes to optimise performance and deliver great results. There are few areas in life where you can start something new and expect it to be perfect right away – is no different.
Why A/B testing matters in referral marketing
As a marketing professional, you will no doubt be used to A/B testing: You take a group of people and divide them into two (or more) groups to test any aspect of a marketing campaign. Your control group will experience Version A and the test groups will see a variation of that.
When you combine this approach with referral marketing software, it enables you to test any aspect of your refer-a-friend programme. Discount type, discount amount and content are just a few of the elements to experiment with.
Here, we take a look at what it takes to make sure A/B testing lets you get the most out of your referral marketing strategy.
Key features of an effective A/B testing platform
While the general concept is easy enough to understand, there are some specifics with referral marketing that it's important to get your head around.
Testing multiple variants simultaneously
For true A/B testing, the campaign for the Control and the Test segments must take place at the same time. This ensures that external factors, such as weather, the changing economy or time of year do not distort the results.
If you run one referral offer and then test a different variant at a later date, you risk attracting groups that may be unrepresentative of your normal traffic and skewing your results.
Testing by customer segment
A/B testing and cohort analysis are usually described as separate things. Cohort analysis is usually defined as studying the data on two separate groups over time.
A/B testing by cohort, or customer segment, brings together the benefits of both strategies. Each cohort receives a different test but within each cohort, everyone has a consistent experience in terms of incentives and content, even over long buying cycles.
Why is this so important for marketers?
No doubt, you’ll want to test different referral offers and messaging all the time. There might be seasonal events which you’d like to incorporate in your referral marketing (extra rewards around Black Friday or Christmas for instance), or you might have other business pressures which mean you have to substitute a different value reward.
Amending marketing offers is fine for new groups of customers. But not for customers you’ve already promised the incentive.
Imagine your favourite brand offering you £40 if you introduce a friend to the company and £40 for your friend. You love the brand and you know your friend is in the market for their product so you’re more than happy to refer them. Now imagine if your friend goes on to buy from them 2 months later, but they only give you both £10. Not a great experience for the newly introduced customer. Even worse, you feel embarrassed for introducing them and your loyalty to the brand disappears like a puff of smoke.
A/B testing by cohort means you can test away to your marketing heart's content, but the offer will always remain the same for each cohort.
Real-time analytics and insights
One of the biggest mistakes people make when A/B testing is stopping too early. This can lead to mistakes in rolling out the “winning” referral test.
To get results you can trust, run A/B tests until you get a big enough sample and ensure that the results are statistically significant. The Mention Me Customer Advocacy Intelligence Platform automatically alerts you when a test reaches significance, but if you are not lucky enough to use our solution then there are many online tools to help your calculations, such as this one from Neil Patel.
Even if your tests reach significance quickly they should be run for a minimum of 2-3 weeks.
Why? You’ll know from experience that email open and click-through rates vary significantly by day. Likewise, on social media, there are optimal days and times to post.
If you are fortunate enough to get enough traffic to your referral programme in a few days you might reach significance, but the results will similarly be skewed by day of the week.
A/B testing ideas for referral marketing
The design of your test programme is all-important. A well-designed A/B test can yield powerfully illuminating insights, while a poorly designed one can see your marketing go off in a completely mistaken direction.
Through referral testing, you’re finding out more about what drives your customers. You’ll need to test what will motivate them to share at different stages of the customer journey and at different points in the buying cycle. Of course, you should also rigorously A/B test the copy and design across devices.
Optimising referral incentives for advocates
Your customers are most likely to refer your brand at the greatest point of delight. Traditionally, referral marketing programmes prompt customers to refer a friend immediately after they place or receive their order. But brands have also had success experimenting with prompting a little later after the customer has had a chance to fully experience and enjoy the product.
For some customers, the most likely time to refer is after they’ve left a 5-star review.
Experimenting with referral messaging and copy
How your offer looks matters. Do your customers prefer short, to-the-point text next to bright images or do they respond better to longer, more playful copy and toned-down imagery?
We have run this test many times, and it has always achieved the same result. A 2x improvement is achieved with the overlay option.
Option 1 - Overlay
Option 2- On Page
Option 3 - Corner Peel
Experimenting with different types of copy demonstrates the need to test your ideas out in practice. Kurt Geiger found that, in a time when short attention spans are often assumed to be the norm, the longer and more descriptive copy emerged as the clear winner of their A/B tests.
Using detailed and descriptive messaging, as opposed to short, concise content, led to a 21% increase in shares and an impressive 98% increase in new customers from referral.
Framing your offer
Do you lead with the offer your customer’s friend will receive if they make a purchase (generous) or with the offer you’ll give the customer for the introduction (self-interested)?
Option 1 - "generous" offer
Option 2 - "self-interested" offer
This appears to perform differently for different brands. Interestingly, in a recent OnePoll survey that we ran, when asked which offer they preferred, the overwhelming favourite with respondents was Option 2, with almost four times as many choosing the "self-interested" option over the "generous" offer.
Testing referral reward type
Testing a % discount versus a £ discount is a popular test and can have a big impact on results.
A recent test by one of our homewares clients saw a 61% increase in customer acquisition with a percentage discount compared to the “£ off” discount.
Our OnePoll research also reflected this result with % discount being the most popular offer, particularly for males, with 39% preferring this to a range of other offers. Females also liked this offer most (29%), but were also interested in free gifts and free delivery.
Depending on the demographics of your customer base then, you could well see a different result.
The benefits of A/B testing in referral marketing
Having run referral programmes for over 450 brands, we know that A/B testing is essential. On average, A/B testing by cohort increases referral customer acquisition by 4x within 6 months.
Maximising customer engagement and conversions
Testing different types of content, offers and messaging means you can, over time, increase your engagement and conversions.
You might not make the impact you imagined when you dreamed up that killer piece of marketing copy the first time around, but if you keep trying out different approaches and monitoring what works best you will arrive at a formula that works.
And what's more, if you do enough testing, you can be sure that the strategy and tactics you're pursuing are the right ones to maximise results.
Optimising referral program performance over time
Customer referral programs are not static; they evolve over time. What worked well in the past may not be as effective today. To keep your referral program performing at its best, continuous optimization is key. A/B testing is a powerful tool for ensuring your program remains relevant and successful.
As your customer base grows and your business expands, it's important to adapt your referral program to meet changing needs and preferences. A/B testing can help you identify which program features or incentives are resonating most with your current customer base.
Gaining valuable insights into customer behaviour and preferences
A/B testing not only improves the performance of your referral marketing but also provides valuable insights into your customers' behaviour and preferences. Through rigorous testing, you can uncover patterns and trends that help you understand what drives your customers to refer your brand to others.
If you want to find out more about how A/B testing can be a part of a referral marketing strategy that turns your customers into enthusiastic promoters of your brand, take a closer look at Mention Me’s groundbreaking Customer Advocate Intelligence Platform.