SRM with automatic traffic allocation


I have a client using Mida. and they’ve been using the option to “Automatically give more traffic to better-performing variants”. Which, from all I’ve read, and experienced in the past, this just introduces SRM and makes the results untrustworthy.

Mida says, " This uses machine learning to learn from data gathered during the test to dynamically increase the visitor allocation in favour of better-performing variants."

Is there a use case for this on features that need to be deployed quickly, or would you not touch it at all?


Last edited by @Jono 2024-02-27T20:24:53Z

Hi Jono,

I think that all depends on how you or your client planned to use the Bandit approach. SRM is important if you are doing A/B testing by the book, so you intend to reject the Null hypothesis, meaning that if the new variant performs better than the original, you want to apply it when the test concludes.

I see two use cases for the Bandit approach (maybe there are more, I link a CXL article that it’s great)

  • To get the most you can for a Time-bound campaign, Advertisement, etc.
  • Predictive Personalization: It is similar to the bandit approach but extended in time. You keep running the experiment with different variations, and the system will learn which variant performs better depending on the traffic that lands on the page.

The CXL course Personalizing for conversion explained the personalization option very thoroughly

I hope that this helps you)

Best regards,