Skip to main content

Global control group

A global control group is a slice of your contacts who receive nothing - no campaigns, no journeys, no marketing of any kind - so you always have an honest answer to the one question no individual campaign report can answer:

Is all of this worth anything?

Every other control group in Joryio measures one campaign or one experiment. This one measures you. Find it under Audience → Global control group.

Why campaign-level results are not enough​

A campaign report compares people who got that campaign against that campaign's own control group. Do that fifty times and you have fifty local answers, none of which add up to a total - the same person appears in many of them, campaigns overlap, and a customer who would have bought anyway gets counted as a win in whichever campaign reached them last.

The global control group sidesteps all of it. Its members are chosen at random, so they are statistically the same as everyone else in every way you can and cannot measure. The difference in what they spend is what your marketing is actually adding.

Turning it on​

SettingWhat it does
EnabledWhether anyone is held out at all
PercentageHow much of your audience is held back. Capped at 20%
Exclusion by tagSends carrying any of these tags reach everyone, holdout included

Membership is stable: the same contacts stay in the group, so the comparison means something over months rather than being reshuffled each send. It is decided by the contact's own ID, not by a stored list, so it costs nothing and never drifts out of sync with your audience.

What the percentage costs you​

Held-out contacts receive nothing, so you are deliberately giving up the revenue they would have produced. That is the price of knowing the number. A 5% holdout on a list that generates 100,000 a year is roughly 5,000 of forgone revenue - and in exchange you learn whether the other 95,000 is real.

Start at 5% if you are unsure. Small audiences need a larger slice to say anything confidently - the same trade-off as any other control group.

Exclusion by tag​

Some messages must reach everyone, whatever the measurement costs. Tag those sends and list the tag here.

Use it for anything a held-out contact would be harmed by missing: a service notice, a policy change, a security message, an order problem. Do not use it to rescue a campaign you think is too important to hold out from - that is exactly the campaign whose value you do not yet know.

Transactional messages (anything sent with subscription preference all) are never held out and need no tag.

What it does not hold back​

  • Transactional messages. Receipts, password resets, order updates.
  • Anything tagged for exclusion, as above.
  • Messages a contact requested, such as a double opt-in confirmation.

Reading the result​

The page shows conversion and revenue for everyone who received marketing against the held-out group, and the difference between them - your incremental revenue.

Two things decide whether to believe it:

  • Significance. Joryio tests the conversion rate, not revenue. Revenue per person is dominated by a handful of large orders, and a significance test on it produces confident-looking numbers that mean nothing. See Statistical significance.
  • Time. A holdout needs enough sends and enough purchases to separate signal from noise. Expect weeks, not days.

How it combines with campaign control groups​

They nest - they do not add up.

A contact in the global control group is excluded from the campaign entirely, so they are never also picked for that campaign's control group. With a 10% global holdout and a 20% campaign control:

GroupOf 1,000 contacts
Global control group100
Campaign control group180 (20% of the remaining 900)
Receive the campaign720

So 280 receive nothing - not 300. If the two were assigned independently, globally held-out contacts would land inside campaign control groups, and every campaign's measured lift would be inflated by people who were never eligible to receive it in the first place.