Far from dying, email seems to grow in both effectiveness and importance each year. More and more digital products include some email component to either onboard new users or as part of the functionality of product. For example, in our design platform, Notable, emails are a critical component of the way the product works. Users are notified when they are added to projects, when feedback is given on design work, and when updates are made. Other popular digital products and services work in much the same way, with email serving many functions core to the product itself. Yep, email is not just a tool for marketers anymore. It’s a tool product designers can use to convey information and drive action.

In this lesson, we’ll break down what the most common email metrics are so we can measure the effectiveness of our emails, and dive into A/B testing so we can continue to iterate on our design and content.

Check the Numbers

Most ESP’s (Email Service Providers) provide some sort of campaign analytics reporting that can both help you track your growth and quickly identify both technical and content problems.

Open Rate:

This is the number of subscribers who have opened your email. Open rates vary wildly across industries, but in general, you want to see at least 20% or better. (Note: The number of opens is not 100% accurate due to the way opens are measured and tracked, but is still useful for getting a general picture of how successful your email is.)

Click Through Rate (CTR)

This is the proportion of people who opened your email and clicked one or more of the links in your email. In general, you should focus on just a few, or preferably one main call to action (CTA) you’d like your user to click on.

Unsubscribes

This is the number of your subscribers that have voluntarily chosen to unsubscribe from your email list.

Bounce Rate

When an email cannot be delivered and is rejected by a subscriber’s email server, it is classified as a bounce. Bounces are split into two categories, hard and soft bounces. A hard bounce is generally more serious and indicates a permanent reason someone could not receive your email. This could be because the email address is no longer valid, the domain name doesn’t exist anymore or you’ve been blocked. A soft bounce usually indicate a temporary delivery issue, things like a full inbox or an offline server. If you see a high number of bounces on your first send, don’t panic, this is normal.

These metrics can help you diagnose all kinds of technical issues. For example:

• Seeing a sharp drop in your open rate or a rise in your hard bounces? You’re emails are not reaching your desired recipients. This could indicate you’re being blocked or being put in the spam bin.
• Not seeing any click through? Something is wrong with your CTA. This can indicate a bad link.

Analytics can also point out content problems:

• Notice that your CTR is low on your main CTA? The link is functional but not effective. This could signal that your CTA is unclear or not compelling enough. You may choose to experiment with the copy, color, size, or location of the CTA and see if it improves your CTR.
• A high number of unsubscribes could indicate a problem with your content and the value you are delivering. If any of these warning flags go off and an investigation confirms your emails are technically sound, it’s time to start experimenting with the design and content of your emails. A/B testing should be employed to help you determine the effectiveness of your changes.

Testing, 1, 2, no 3

A/B testing is comparing two different versions of an email to see which one performs better. To do this effectively, only one element should be changed, this is called a variable. By testing only one variable at a time, you will be able to more accurately measure the impact of your changes. You also need to decide what the winning criteria for your test is. Are you aiming for higher open rates, higher clickthroughs or some other goal? Be sure to keep track of your tests and their results so you can draw insights that can shape your strategies and tactics moving forward.

There are all kinds of variables you can test:

Subject Line

The subject line is one of the most powerful factors that influence people to open your email. Shorter subject lines tend to work better for most industries, but experiment with both length and style to see what works best for you. Determine the effectiveness of two different subject lines by comparing open rate.

Sender Name

You should always be honest about who you are, but you may want to try either displaying your personal name or your company/product name to see what proves most effective. Determine the effectiveness of two different sender names by comparing open rate.

You can test how both your subject line and sender name look on mobile devices using our free tool, TestSubject

Send Time

The highest number of email opens happen early in the day and midway during the week, but you may find other times that work better for you. Try sending your emails on different days and at different times. You can measure the effectiveness of this by comparing both open rates as well as CTR. You may find that you may get lower opens but higher CTR sending in the evenings or on weekends.

CTA

The appearance or copy of your CTA can greatly affect the level of engagement with it. Try different colors, the use of text links vs. buttons, different locations, and variations of copy to see what is most effective. Determine the effectiveness of two different CTAs by comparing click through rate.

Personalization

Studies have shown that tactful use of personalization, or using your recipient's name, location or other personal data, can increase both open rates and CTR. Experiment with including personalization in your subject line and in the body of your message. Determine the effectiveness of using personalization by comparing open rates and CTR between a version with it and one that does not include any personalization.

---