Splits

Overview

Audience Splits enables you to create randomized audience segments for accurate, controlled testing of campaign strategies. By splitting an audience into distinct groups, you can compare outcomes between different treatments, channels, or creatives, gaining clear insights into what's truly driving your campaign performance.

What you can measure

• Incrementality: Did your campaign actually cause the desired outcome?
• A/B/n testing: Compare different creatives, messages, or channels.
• Personalization impact: See if personalized experiences outperform generic ones.
• Channel effectiveness: Identify which channels drive more downstream conversions.

When to use splits

Use audience splits when you want to move beyond assumptions like:

• Correlation: “Purchases increased after my campaign—but did the campaign cause it?”
• Attribution: “A user converted after seeing my email—but would they have converted anyway?”

Running a split test helps you isolate the effect of your campaign by comparing outcomes between randomized groups. This gives you a clearer view of what’s actually driving results.

How audience experiments work

Example use cases

Email campaign holdout

Goal: Test whether sending an email increases purchases.

• Split audience 50/50
• Send promo to Group A, hold out Group B
• Measure conversions or revenue per user

Ad creative A/B test

Goal: See which ad performs better.

• Split audience into two groups
• Sync to separate ad sets in Meta or Google
• Compare CTR or conversion rates

Personalization test

Goal: Measure the impact of tailored content.

• Group A: Personalized product recs
• Group B: Generic message
• Measure engagement, clicks, or sales

Lifecycle messaging experiment

Goal: Improve onboarding outcomes.

• Split new users at signup
• Sync to two onboarding flows
• Measure feature activation or retention after 14 days

Channel comparison

Goal: See which marketing channel performs best.

• Sync Group A to email, Group B to paid ads
• Measure downstream conversion or LTV

Setup steps

1. Split your audience

1. Open your audience in Customer Studio.
2. Go to the Splits tab.
3. Toggle Enable split groups.
4. Under Split audience into groups, define your split:
   • Set the number of groups (e.g., 2 for A/B tests, 3+ for A/B/n tests).
   • Adjust the percentage distribution between groups (e.g., 50/50).
   • Rename groups as needed (e.g., "Holdout", "Treatment A").
5. Optionally add more groups for A/B/n testing.
6. For each group, configure syncs to destinations.
   • For example: send the treatment group to Meta Ads and the holdout group to a suppression list.
7. Click View Results to begin analysis.

2. Measure results

Clicking View results will open a Splits chart type.

Configure the splits chart →

Holdout group logs

Holdout group logs track which rows were excluded from a sync as part of a split group (e.g., a 20% holdout). These logs help data teams run detailed post-campaign analysis in the warehouse, such as identifying exactly who received a treatment and who didn’t.

How to enable holdout group logs

1. Contact Hightouch to enable the Holdout group logs feature flag. This requires:
   • The Lightning sync engine
   • A destination with write access to your data warehouse
2. Toggle the setting in workspace sync log settings
   • Go to Integrations → Sources and select a source
   • Navigate to the Sync Logs tab
   • Check the box for Audience holdout group logs

When enabled, Hightouch writes excluded rows to your warehouse in the hightouch_audit.audience_holdout table. Each record includes the split_group, row_id, timestamp, and a snapshot of the source data. Rows that were synced are stored separately in the sync_changelog table—both must be joined for complete analysis.

Table definition

The hightouch_audit.audience_holdout table is an append-only log. If the same row is included in multiple sync runs, it has multiple entries in this table.