Segment A/B test results

Contributors

@elise-dopson


Business Benefits

Understand the different behaviors on your cohorts.


Segment your test results once the test has concluded and the test is statistically significant.

This way, your segmentation has more validity than during the test or with an inconclusive hypothesis test.

Take action or gain useful insights based on the segmentation.

If your test was a part of a lead generation or sales test, then it’s more useful to have a segmentation strategy. Oftentimes, tests are run that are higher up in the funnel and segmentation isn’t as actionable, which is OK, since there are learnings that come with that.

Figure out if your hypothesis aligns with any obvious segmentation.

Your segmentation could be staring you right in the face. Does your testing hypothesis warrant an obvious segmentation? For example, Our hypothesis is that mobile users want to watch a video on the sales page instead of reading a lot of text due to the high bounce rate. Segmenting your traffic by device type is essential to your segmentation analysis in this case.

Start with the most common segmentation types to use in your marketing, like return visitors vs new visitors, device type, traffic source, or where in the funnel they are.

Send data to your CRM and ad platforms so that you can take action based on your testing data.

In order to be more actionable in your testing, implement a tagging strategy so that the results are easy to also segment in your CRM, where you will send follow-up emails. The other way to be actionable is to make sure you are creating audiences and using pixels, based on your ad platform, to re-target those segmented audiences, if applicable.

Store your results in a testing and learnings spreadsheet to keep the learnings in one spot.

Just like annotations in Google Analytics, proper documentation of the tests and learnings can really help with long-term marketing optimization. You won’t always have an immediate takeaway. Reviewing the test, learnings, and outcomes over time can help show trends and patterns as to the overall audience and what works or doesn’t work.

Last edited by @hesh_fekry 2023-11-14T16:20:22Z