Set up A/B testing on email campaigns

Contributors

@sonja4dp-com-au


Business Benefits

Improve your open, click-through, and conversion rates.


Decide on your goal for the testing process.

For example:

  • Increase website traffic
  • Improve opening rate
  • Increase conversion rate
  • Lower the bounce rate
  • Lower card abandonment

Decide which aspect of your campaign to test, like UX, design, or wording.

Pick a specific element related to the campaign aspect you picked, like the subject line, the call to action, or formatting.

If you’re having difficulty with people opening your emails, start by improving your subject line. A good subject line will skyrocket your open rates. Continually split test them.

Create two versions of your email, with only one element changed.

Divide your entire list in half and send the original to 50% of the list and the variation to the remaining 50%.

Many email automation tools like Reply.io or Mailchimp have features to split email traffic, so each variation will get a random sampling of visitors. This helps you identify which variation performs better.

The more people you send your tests to, the more accurate it will be, and the more radical the changes between versions are, the more significant the results should be.

If you have a limited-time offer, send a split test to a smaller amount of your list to see which version performs better before sending the best performer to the rest of your list.

Determine the best performer and use that version.

Remember - sample size is not as important as the significance of the variation you are testing. The more radical the change, the more important is the outcome.

Make sure you are confident that the winning version will have a significant effect on what you set out to do.

Check your results and analyze data on open rates, click-through rates, and conversions.

Many email providers like MailChimp, Campaign Monitor, Hubspot, Keap, and Active Campaign provide split testing results as part of their services. Here are some things to look for:

  • If you’re testing your subject lines, look for increased open rates.
  • If you chose BUY NOW vs CLICK HERE, look for the number of people who clicked one versus the other.
  • Check click-through rates to see which style was opened more frequently.