Talk to Sales
Talk to Sales

How AB testing can make or break your referral marketing programme

Angela Southall
By Angela Southall — October 2, 2017

The most successful brands we work with continually split test elements of their referral programmes to optimise performance and deliver great results. There are few areas in life where you can start something new and expect it to be perfect right away – referral marketing is no different. 

As a marketing professional, you will no doubt be used to AB testing; where you take a group of people and divide them into two (or more) groups to test any aspect of a marketing campaign. Your control group will experience Version A and the test groups will see a variation of that.

In the referral marketing arena, this enables you to test any aspect of your refer-a-friend programme. Discount type, discount amount and creative being just a few of the elements to test.

Although that sounds quite straightforward, the reality is that in referral marketing it can be more complicated that you expect. The time between a friend recommending a brand and their friend making a purchase can sometimes be quite long. But to be a success, and to ensure a great customer experience, you’ll need to ensure the friend receives the reward they were promised, even if that is months later.

Here we take a look at the main things to ensure AB testing delivers referral success for your brand.


3 key ingredients of an AB testing platform

3 Key ingredients of a referral AB testing platform

1. Tests run simultaneously

For true AB testing, the campaign for the Control and the Test segments must take place at the same time. This ensures that external factors, such as weather, economics or seasonal factors do not distort the results.

The risk of running one referral offer and then testing a different variant at a later date is that the groups may be unrepresentative of your normal traffic and skew your results.


2. Testing is by cohort

AB testing and cohort analysis are usually described as separate things. Cohort analysis is usually defined as studying the data on two separate groups over time.

AB testing by cohort brings together the benefits of both strategies. Each cohort receives a different test but within each cohort everyone has a consistent experience in terms of incentives and creative, even over long buying cycles.

Why is this so important for marketers?

No doubt, you’ll want to test different referral offers and messaging all the time. There might be seasonal events which you’d like to incorporate in your referral marketing (extra rewards around Black Friday or Christmas for instance), or you might have other business pressures which mean you have to substitute a different value reward.

Amending marketing offers is fine for new cohorts of customers. But not for customers you’ve already promised the incentive.

Imagine your favourite brand offering you £40 if you introduce a friend to the company and £40 for your friend. You love the brand and you know your friend is in the market for their product so you’re more than happy to refer them. Imagine if your friend goes on to buy from them 2 months later, but they only give you both £10. Not a great experience for the newly introduced customer. Even worse, you feel embarrassed for introducing them and your loyalty to the brand disappears like a puff of smoke.

AB testing by cohort means you can test away to your marketing heart's content, but the offer will always remain the same for each cohort.


3. Your referral platform should provide real time statistics so you can see when tests can be rolled out

One of the biggest mistakes of AB testing is that the tests are stopped too early. This can lead to mistakes in rolling out “winning” tests.

To ensure that you make the right decisions you should run AB tests until you get a big enough sample and ensure that the results are statistically significant. The Mention Me referral platform automatically alerts you when a test reaches significance, but if you are not lucky enough to use our solution then there are many online tools to help your calculations, such as this one from Neil Patel.

Even if your tests reach significance quickly they should be run for a minimum of 2-3 weeks.

Why? You’ll know from experience that email open and click through rates vary significantly by day. Likewise, on social media there are optimal days and times to post.

If you are fortunate enough to get enough traffic to your referral programme in a few days you might reach significance, but the results will similarly be skewed by day of week.  


Interesting AB tests to try

Designing a test programme is where the real creative thinking power is put to the test.  Consider elements like the psychology of the brand’s customers, what will motivate them to share, the perceived value of a discount compared to a free gift or other types of rewards. Of course, you should also rigorously AB test the copy and design across devices.

Where to engage your customers

Your customers are most likely to refer your brand at the greatest point of delight. In most cases this is when they place or receive their order. It makes sense then to test how you display your referral offer at this point to maximise conversions

We have run this test many times, and it has always achieved the same result. A 2x improvement is achieved with the overlay option.

  Option 1 - Overlay  

Option 1 - Overlay

  Option 2- On Page 

Option 2 - on page

  Option 3 - Corner Peel

Option 3 - Corner Peel

Framing your offer

Another great test is how you frame the offer. Do you lead with the offer the friend will receive if they make a purchase (generous) or with the offer you’ll give the customer for the introduction (self-interested)?

Option 1 - "generous" offer

Generous refer-a-friend offer

Option 2 - "self-interested" offer

Self interested referral offer


This appears to perform differently for different brands. Interestingly, in a recent OnePoll survey that we ran, when asked which offer they preferred, the overwhelming favourite with respondents was Option 2, with almost four times as many chosing the "self-interested" option over the "gererous" offer.


Reward Type

Testing  a % discount versus a £ discount is a popular test and can have a big impact on results.

Referral reward types

A recent test by one of our homewares clients saw a 61% increase in customer acquisition with a percentage discount compared to the “£ off” discount.

Our OnePoll research also reflected this result with % discount being the most popular offer, particularly for males, with 39% preferring this to a range of other offers. Females also liked this offer most (29%), but were also interested in free gifts and free delivery.

Depending on the demographics of your customer base then, you could well see a different result.


The upside of AB testing

Having run referral programmes for over 450 brands, we know that AB testing is essential. On average, AB testing by cohort increases referral customer acquisition by 4x within 6 months.

More referral AB tests to try

You might also enjoy

Black Friday 2022
Seasonal promotions -

Read time: 4 mins

With 150 days to go, here’s what’s missing from your Black Friday planning

Black Friday 2022 is the perfect chance to prove to your customers why your brand is worthy of their cash — and to encourage them to return after the event.

Read time: 3 mins

How Mentioneers are celebrating Pride 2022

To celebrate Pride, we asked Mentioneers what the month means to them. Here's what they said.

Referral marketing -

Read time: 3 mins

Global recession + high inflation = a different marketing approach

Wandering the massive floor at VivaTech in Paris two weeks ago required a self-imposed cognitive dissonance; a healthy dose of denial. 

Stay in the know

Subscribe to our blog and get monthly emails packed full of the latest marketing trends and tips