I have a client using Mida. and they’ve been using the option to “Automatically give more traffic to better-performing variants”. Which, from all I’ve read, and experienced in the past, this just introduces SRM and makes the results untrustworthy.
Mida says, " This uses machine learning to learn from data gathered during the test to dynamically increase the visitor allocation in favour of better-performing variants."
Is there a use case for this on features that need to be deployed quickly, or would you not touch it at all?
I think that all depends on how you or your client planned to use the Bandit approach. SRM is important if you are doing A/B testing by the book, so you intend to reject the Null hypothesis, meaning that if the new variant performs better than the original, you want to apply it when the test concludes.
To get the most you can for a Time-bound campaign, Advertisement, etc.
Predictive Personalization: It is similar to the bandit approach but extended in time. You keep running the experiment with different variations, and the system will learn which variant performs better depending on the traffic that lands on the page.
