A control group is a set of users deliberately held out from an ad, an email or a campaign during a test. It serves as the comparison baseline: the difference against the treated group is the real effect of what was done. Any increase measured without a control group blends with seasonality, campaign and price effects.
Size is critical. Too small a control group is statistically meaningless, too large is unnecessary lost revenue; in practice a small slice of the audience is held out and the test period is extended according to volume. The second rule is random selection; putting the least active customers in the control group breaks the test from the start. Keeping a permanent holdout slice in email and WhatsApp automations makes the claim that the automation works verifiable all year round.
Size is critical. Too small a control group is statistically meaningless, too large is unnecessary lost revenue; in practice a small slice of the audience is held out and the test period is extended according to volume. The second rule is random selection; putting the least active customers in the control group breaks the test from the start. Keeping a permanent holdout slice in email and WhatsApp automations makes the claim that the automation works verifiable all year round.
Size is critical. Too small a control group is statistically meaningless, too large is unnecessary lost revenue; in practice a small slice of the audience is held out and the test period is extended according to volume. The second rule is random selection; putting the least active customers in the control group breaks …
Knowing the term is not enough; you need to know which lever to pull. In a free 30-minute call we look at your store's actual numbers.
Get a free analysis