Master Guide To Split Tests

Sending marketing materials to a mass audience is always effective. However, you’re often leaving incremental revenue out of your formula if you don’t know what works best.

May 22, 2025

Sending marketing materials to a mass audience is always effective. However, you’re often leaving incremental revenue out of your formula if you don’t know what works best. Even more dangerous, you’re going to lose engagement and run into problems with how you’re sending as your email channel grows.


You will find massive differences when you have larger sends with little to no data with approaches that use guidelines and testing methodologies to define audiences and best marketing practices. Sending with higher frequency to a full audience, despite what many think, doesn’t equate to more success and revenue as a business.


The sophisticated strategies of split testing allow you to get the most engagement and revenue from your list while knowing when to send the right message to the right audience.


Finding the Right Mechanics to Your Split Tests

There are different methodologies to split tests to get the results you’re looking for. Before you start, know what you are aiming for - performance, learning, or optimization. Knowing your objective for growth is the foundation to building and scaling a successful email channel.  


Listen and Learn:

This is a no touch formula to help you with testing. You simple split parts of your email sends in 2 or 3 versions and gather results. By doing this over a longer time period, you will find results with what works best for a given test. If you want to understand audiences, engagement, and deeper parts of your ESP, this is the version to use.


Performance:

Most ESPs allow you to define performance metrics. These tests are set up as a 20 / 80 or 10/90 split. Whatever version has the best results with a smaller percentage of audience members will then equate to the winning version sending to the rest of the audience. This way, you don’t waste email real estate on learning, but instead, get the most traction with each send based on those performance metrics. These are best to use with smaller tests where you want to build more momentum and traction.


Optimization:

Similar to the listening tests, optimization lets you define what works and what doesn’t. You then make adjustments in your next sends and test it again with another hypothesis. This methodology works best for smaller changes, such as design and content direction.


How to Lay Out a Split Test

Email marketing, when perceived from the depth of the craft, is a combination of creativity, performance, and science. Split tests are the lab of email marketing and make up the learnings to become more innovative and more calculated about each send. When you build out your split tests, follow specific formulas to make sure you are getting the right measurements in return.

  1. Objective based testing.

    If your split test isn’t moving the needle with your email channel, then it is testing for the sake of testing. Have a larger objective that you are trying to reach in order to reach your objective and to find the results needed.
  2. Hypothesis.

    There is no true test without a hypothesis. You want to have a question in mind. What are you trying to solve? What do you want to know about your audience? What are you most interested in learning in the moment? What do you want to bet against to grow your email channel?
  3. Identify Your Control.

    You should always be testing against something for your results to be clear. Set a control to work against. For instance, if you are testing creative, the original creative vs. a specific question in your hypothesis becomes the test. If you are working in an automation, then the current, running email is the control you are testing against. For tests to work, you need to have a “what is” vs “what could be” set up.
  4. 1:1 Tests Work Best.

    If you have several tests running at once, then your results will skew. Test 1 control vs. 1 new concept to get best results. This will help to identity sources of truth. If it’s identifying a larger part of the channel, run the same test several times to make sure it works.
  5. Expected Results vs. True Results.

    Expected results are your game of chess. This is where you get to gamble by identifying what you think the audience will do. If your true results don’t produce the same results, then it gives you the option to bring up the bigger, related issues of understanding audience responsiveness and what works. Expected results let you know that there is something missing with your audience and how you relate to the channel.


What You Must Know! Identifying Cornerstone Learnings

Just starting to understand your channel? Use split tests to build your best practices and to identify what you want to keep as a “must do” for your channel. Here’s your pillar split tests to work with:

1. Timing. The email that sends at 6 PM at night on Friday vs. the email that sends Tuesday morning at 10 AM have different results. While there are standard best practices, every audience is different. Get to know them by when they are the most responsive. There are three methodologies you want to test timing with:

* Time of the Week
* Morning vs. Afternoon
* Time of the Day


Start with the weekdays that are best to send by. Then, break this down into whether those days have a better morning or evening send (10 AM vs. 2 PM). Keep this standard. After you see these metrics, get specific. If 10 AM is better, then it means it’s a morning definition. Refine the test to 8, 9, 10, and 11 AM to determine the absolute best time.

2. Content. Does your audience like informational blogs or UGC? While every email should have different types of content, you also want to see what creates different types of engagement. The more you calculate this part of your email, the better it becomes.

Content also relates to length and amount of engagement time. Long vs. short tests are essential in getting results from your audience.

3. Design. Lifestyle images vs product images? Stickers vs. CTAs? Editorial vs. product grid? Design has several elements that create responsiveness. Know where your audience engages and how they relate to define the design that offers the most.


Next Level Marketing - Advancing Your Split Tests

Science rules when you are working with advanced split tests. This is where you can let the lab rat come out of your personality. Advanced split tests guide and direct the impact of the ESP, allowing you to have more room for growth.

These are the tests that don’t set best practices but give you a deeper understanding of how to play with your email channel. Design and content can move into more advanced methodologies to split; however, you usually want to get into personalization and audience responsiveness to use advanced tests for targeting.

  1. Audience intent to purchase.

    Test your non-purchasers by RFM (recency, frequency, monetization value). Have they been inactive for a long time and receiving 8 emails a month? Change the frequency to 1x per month and see what activation looks like.
  2. Purchaser personalization.

    If you’re treating purchasers the same way as new subscribers, then you’re missing out on opportunities. Testing purchaser behaviors by the type of content they receive will support better results with the type of sends you are putting together.
  3. Testing the customer lifecycle.

    The customer journey doesn’t just include one action they take. Test your audience at a deeper level by defining what their behavior is over time. This type of test is a great opportunity to go across channels. If prospects are active in social and then on the site, test what type of experience they want to have. If they are new, determine different tests to optimize against vs. those that are active or returning. Follow your audience to determine best concepts to send by their placement in the lifecycle.
  4. Personalization.

    A much deeper way to test is to look at ways to personalize. In best instances, this happens with dynamic blocks that target specific audiences to see how they respond. Personalization can also be tested by RFM. For instance, if someone is in the browse abandon, personalize their experience by how many times they browse, how often they visit, when the last visit was, and relate that to email. See if a change in messaging changes how they interact with your channel.


How Much Is a Split Test Worth?


At the base level, every ESP channel should have their defined best practices that they carry with them. These can be retested twice a year to support growth of the channel, but keeping the formulas of best sending practices alive is reliant on the channel testing.

Split tests beyond the basics are the main incremental driver of growth. It is the centerpiece to a channel that is able to scale. It helps in understanding audiences, it builds personalization, and it lets you know exactly how to target.


There are two ways to grow an email channel. One is by sending a lot to a lot of people. While you will see revenue as a result, it will ultimately start to lead to inundation and fatigue within the channel. The second is by targeted sending. The only way to target is to test. Testing and optimization is a centerpiece of best email practices, allowing you to become the green line that shoots to the next level of scalability.

Related articles

See all

Any project in mind?
Get in touch with us

Join us as a strategic partner and take your brand potential to new heights.

WORK WITH US

© 2025 Structured. All Rights Reserved.