Running multiple tests for the same product skews test data for both tests making it impossible to know which test variable helped or hurt the overall test performance. Instead, Optimal A/B requires that you only test one variable at a time for each product, leaving all other variables the same.
“In an experiment, scientists only test one variable at a time to ensure the results can be attributed to that variable. If more than one variable is changed, scientists cannot attribute the changes to one cause.”
-Google AI
