A/B testing

Why A/B testing matters

A/B testing (also called split testing) lets you send two versions of a campaign to different portions of your list to determine which performs better — higher open rate, more clicks, or more conversions. Instead of guessing which subject line, CTA, or content approach works, you measure it.

The most commonly tested elements are subject lines (highest impact on open rate), from names, preview text, email content, and send time. Even small improvements compound significantly across many sends.

Manual A/B testing with segments

The segment-based approach works by splitting your list into two groups — Segment A and Segment B — and sending a different campaign variant to each. Here is how to set it up:

1. 1
   Create a split field on your subscribers

   Add a custom text field called "ab_group" to your subscriber schema (Settings → Custom Fields). When you import subscribers or add them via API, assign each one to either "a" or "b". For an even split, alternate: first subscriber gets "a", second gets "b", and so on. You can also do a 20/80 split — send variant A to 20% of your list to test before sending the winner to the remaining 80%.

2. 2
   Create two segments

   Go to Subscribers → Segments → New Segment. Create "Variant A Group": condition is custom field "ab_group" equals "a". Create "Variant B Group": condition is custom field "ab_group" equals "b".

3. 3
   Build two campaigns

   Create Campaign A with your first variant (e.g. subject line A). Create Campaign B with your second variant (e.g. subject line B). The email body can be identical — only change the element you are testing.

4. 4
   Send both campaigns simultaneously

   Schedule both campaigns for the same send time. This controls for time-of-day effects. Campaign A sends to Segment A; Campaign B sends to Segment B.

5. 5
   Compare results

   After 24–48 hours (enough time for most opens to occur), compare the open rate, click rate, and any conversion metrics between the two campaigns. The variant with better performance is your winner.

6. 6
   Send the winner to the rest of your list

   If you used a 20/80 split, create a third segment for subscribers who did not receive either test variant and send the winning campaign to them.

What to test

Test one element at a time so you know what caused the difference. Good tests to start with:

• Subject line

  The most impactful variable. Test different lengths, questions vs statements, personalisation (with first name vs without), urgency vs curiosity.

• From name

  Personal name ("Alice from Acme") vs brand name ("Acme Newsletters"). Personal names often improve open rates with colder lists.

• Send time

  Morning vs afternoon, weekday vs weekend. Segment A gets Tuesday 9am, Segment B gets Tuesday 2pm.

• CTA button copy

  "Download now" vs "Get the guide" vs "Start free". Test which call to action drives more clicks.

• Plain text vs HTML

  A simple plain-text-style email (no images, minimal formatting) vs a designed HTML email. Plain text often wins for relationship-style emails.

Planned native A/B testing

When native A/B testing ships in Mally, it will include:

• Automatic list splitting by percentage at send time
• Test window configuration (auto-select winner after N hours)
• Statistical significance indicator
• Automatic send of winning variant to remaining subscribers
• Multi-variant testing (A/B/C)
• Subject line, preview text, and content variant support

Have feedback on what you need from A/B testing? Email us — customer input directly shapes the roadmap.