Determine how your goals align with different test methods.
Write a list of goals you want to accomplish, or elements you want to test, and determine if these are paradigm-changing or not.
For example, if you are testing something that will impact your business model or the future of the company, it is best to test with multiple, separate tests (MST).
Running MST is the best strategy if there is little overlap. Avoid this method if there is huge overlap between tests.
If there is, reduce the number of simultaneous tests, and/or let the tests run longer for improved accuracy. Avoid MST if you suspect extreme interactions.
- If you are running tests to measure the same goal, for example, a purchase.
- Elements in the same flow, for example, running tests on each of your multi-step checkout steps.
- Suspect strong interaction between tests.
With this method you are ensuring that people won’t be part of more than one test.
An MET strategy is complex, and may slow down testing. If you plan to run tests for the same duration, combining multiple tests into one MVT is the best strategy.
Then use this data to invest more time and re-test important elements with multiple smaller, slower, more complex tests.