Testing two subject lines

Instead of guessing which subject works, let your list tell you. On the Schedule step, tick Test two subject lines, write a second subject, and choose the size of the test groups and how long to wait.

What happens

1. 1

   Two test groups

   A random 10% (or the size you chose) get your subject. Another random 10% get the second one.

2. 2

   The rest wait

   Everyone else is held back and shown as "waiting for the winner".

3. 3

   After the wait

   Once the test emails are out and the wait (say 4 hours) has passed, the subject that was opened by more people wins.

4. 4

   The winner goes to the rest

   The held-back people are released and get the winning subject. Their emails go out under your normal limits.

What it needs

• Open tracking turned on. Opens are the only signal that arrives within hours.
• At least 20 people. Below that the groups are too small to mean anything, so the test is skipped and everyone gets your first subject. The Review step tells you.
• A second subject that actually differs from the first.

Where it can mislead you

• Apple Mail pre-loads images, which counts as an open even if the person never looked. It inflates both subjects, so it blurs the difference but doesn't favour one.
• Small groups are noisy. A winner picked from 25 opens each can easily be luck. A tie, or no opens at all, keeps subject A.
• Opens measure the subject, not the email. A subject that gets more opens but fewer replies isn't necessarily better.

If something looks wrong

• The test option is greyed out: turn on "Track opens and clicks" first.
• The rest of my list hasn't been sent: the wait hasn't passed yet, or some test emails are still waiting for a daily limit. The campaign page says "Subject test running".
• I stopped the run during the test: the held-back people are cancelled with everyone else still waiting.
• Everyone got subject A: the list was under 20 people, or subject B was identical to A.