A/B Tests
A/B tests live inside campaign workflows. Add an A/B split node, give each variant a branch with its own message (a different subject, template, send time or channel) and choose how the winner is measured.
How contacts are split
- Each contact is assigned to one variant by a stable hash of contact, campaign and split node, following the configured weights. Re-running a contact through the flow keeps the same variant.
- Saving the workflow keeps an A/B test record per split node in sync (name, variants, weights, winner metric, auto-select, minimum sample and duration).
- Once a winner is declared, every contact that reaches the split takes the winning branch. Contacts assigned earlier keep their variant.
- The dry run shows which branch a contact would take, including the declared winner.
Results
Open a campaign and go to Analytics → A/B tests. For every split and variant you get:
| Metric | How it is counted |
|---|---|
| Assigned | Contacts that reached the split and were routed to the variant |
| Recipients | Unique contacts that were sent at least one message from the variant’s own branch |
| Unique opens / open rate | Recipients who loaded the tracking image, divided by recipients |
| Unique clicks / click rate | Recipients who clicked a tracked link, divided by recipients |
| Click-to-open | Unique clicks divided by unique opens |
| Conversions | Assigned contacts that fired the campaign goal event within the attribution window after assignment |
| Revenue | Won deals of assigned contacts closed after assignment (shown for the REVENUE metric) |
| Lift vs control | Relative difference of the winner metric against the first variant |
| Confidence | 1 − p of a two-proportion z-test against the control |
Expanding a variant shows each message step in its branch (sent, delivered, opened, clicked, bounced, unsubscribed, failed) and clicks per link.
Messages on nodes shared by several branches (for example a common follow-up after the branches merge) are not attributed to any variant.
Winner selection
- Manual: click Declare winner on a variant. Reset winner splits new contacts again.
- Automatic: enable Auto-select winner on the split node. The leader is selected when every variant has reached the minimum sample and the difference is significant at 95%, or when the test duration ends (the clock starts with the first assignment).
- Metrics:
OPEN_RATE,CLICK_RATE,CONVERSION(needs a campaign goal) andREVENUE. Revenue has no significance test, so automatic selection for it only happens when the duration ends.
API
GET /api/workspaces/{workspaceId}/campaigns/{campaignId}/ab-tests
Returns { tests: AbTestResult[], generatedAt } with the per-variant numbers above, the current leader (with pValue and significant), status (not_started, collecting, ready, winner_selected) and the declared winner. Automatic winners that are due are applied before responding.
POST /api/workspaces/{workspaceId}/campaigns/{campaignId}/ab-tests/{nodeId}/winner
Auth: Session, role MEMBER or higher.
{ "variantKey": "var-a" }Use null to clear the winner.
Legacy: GET/POST /api/workspaces/{workspaceId}/abtests
Lists or creates standalone A/B test records. Workflow splits create and update their records automatically, so new tests should be built with the A/B split node.