December 17, 2024
The Definitive Guide To Benchmarking Your Marketing
.png)
Without data, you’re just another person with an opinion.
~W. Edwards Deming, Data Scientist
Marketers are obsessed with measurement. One of the most frequent questions marketers ask is probably, “How did we do vs benchmarks?”
The great thing about looking for benchmarks is that they are abundantly available, and mostly free. With the advances in AI tools like ChatGPT, benchmark data is even more readily available than ever before. On the other hand, because they are abundantly available, how do you know which ones to use?
The great thing about looking for benchmarks is that they are abundantly available, and mostly free. With the advances in AI tools like ChatGPT, benchmark data is even more readily available than ever before. On the other hand, because they are abundantly available, how do you know which ones to use?
Why Benchmark?
Given a choice, I would always recommend using your own historical performance over an industry benchmark as a baseline for comparison.
However, there are situations where benchmarks are meaningful:
However, there are situations where benchmarks are meaningful:
- If you are just starting out with a new type of tactic, and don’t have historical results;
- If you are starting with a new organization in anew industry and need to assess the historical performance;
- If there is a change in external factors, such as the COVID-19 pandemic.
Challenges with Benchmarking
A simple search can give you several benchmark reports with widely different numbers. Suppose you want to find email benchmarks. It’s not uncommon to see open rates that range from the low teens to the high 30s. While the metrics may be calculated the same way, one must remember that there are inherent biases built into these reports.
For email benchmarks, these reports are usually published by email service providers (ESPs) or marketing agencies based on their own clients’ data or survey data. Usually, ESPs and agencies specialize in certain segments (industry, organization size, etc.). So data published by an ESP known for working with smaller businesses can be very different from those published by an agency that works with global brands, even within the “same” industry vertical.
For email benchmarks, these reports are usually published by email service providers (ESPs) or marketing agencies based on their own clients’ data or survey data. Usually, ESPs and agencies specialize in certain segments (industry, organization size, etc.). So data published by an ESP known for working with smaller businesses can be very different from those published by an agency that works with global brands, even within the “same” industry vertical.
Picking which benchmarks to use is, unfortunately, often a subjective decision.
If you are using an AI assistant like ChatGPT to curate benchmark data for you, how do you know what got picked is right for you?
RI Benchmark Distillery™
Enter the Benchmark Distillery™, a benchmarking methodology we have created to help our clients come up with the most appropriate benchmarks for their needs.
The goal of the Benchmark Distillery™ is to help you make an informed, deliberate decision about which benchmarks to use. The methodology can be used for pretty much anything where you can get your hands on a variety of industry benchmarks.
There are 4 steps that lead you to your benchmark nirvana. We will walk through these steps using email benchmarks as the illustration.
There are 4 steps that lead you to your benchmark nirvana. We will walk through these steps using email benchmarks as the illustration.
1. Scour for benchmarks
This is the time to flex your Googling muscles. Searching for common terms like “email benchmarks” and “email open rate industry average” should lead you to many results. Save the results of what you find — download the PDF, print the web page to a PDF, or save the data table as an image. Also save the link to the source page. You never know when you might need to refer back for additional information.
This brings us back to the issue of bias that we talked about before. If you are not familiar with the source publishing the data, this is your chance to learn about them. Is it an ESP or agency? Do they work mostly with certain industries, certain organization sizes, certain geographic regions, etc.? How are they defining their metrics?
Sometimes you will encounter articles with averages and benchmarks. These are useful for finding reports that may not show up in your search. Make sure you trace back to the source to understand how the data is collected and calculated. If the source is not quoted, don’t use the number. I have seen articles cite an industry average as a fact, only to discover that it was from a study done over 10 years ago!
Don’t forget to add your own historical benchmarks if you have some. The best benchmark is often your own, since it’s representative of your audience.
This brings us back to the issue of bias that we talked about before. If you are not familiar with the source publishing the data, this is your chance to learn about them. Is it an ESP or agency? Do they work mostly with certain industries, certain organization sizes, certain geographic regions, etc.? How are they defining their metrics?
Sometimes you will encounter articles with averages and benchmarks. These are useful for finding reports that may not show up in your search. Make sure you trace back to the source to understand how the data is collected and calculated. If the source is not quoted, don’t use the number. I have seen articles cite an industry average as a fact, only to discover that it was from a study done over 10 years ago!
Don’t forget to add your own historical benchmarks if you have some. The best benchmark is often your own, since it’s representative of your audience.
2. Shortlist relevant benchmarks
Now that you have a trove of data, it’s time to organize it.
The easiest way is to track all the data in an Excel spreadsheet/Google Sheet. When you go through the various reports, note down the metrics that you feel are relevant for your organization.
Here is what the list would look like if you were gathering benchmarks for a non-profit. You can see the numbers vary quite widely.
The easiest way is to track all the data in an Excel spreadsheet/Google Sheet. When you go through the various reports, note down the metrics that you feel are relevant for your organization.
Here is what the list would look like if you were gathering benchmarks for a non-profit. You can see the numbers vary quite widely.
Figure 1. Spreadsheet with relevant email benchmarks for non-profits
Some notes of interest:
- Discard reports that are more than 2-3 years old.
- Sometimes you may need to calculate a metric. For example, most studies only show open rate and clickthrough rate. If you want to benchmark your click-to-open rate, calculate it by dividing the clickthrough rate by the open rate.
- We have included some studies that are not specific to non-profit. Depending on the nature of the emails, sometimes the industry vertical may not matter as much. For example, if you have an audience who sign up for a newsletter, whether you are a non-profit or not, you may want to look at benchmarks around newsletters. You may also want to do this if your industry is not a common one, and there is not much data to go by.
3. Set the most applicable benchmarks
Once you have the shortlist, it’s time to select the benchmarks you want to use.
First, find the range of benchmarks, and then determine one that is most applicable for your situation. Here is what I have selected for the Non-Profit benchmarks.
The range is useful if you are trying to estimate the impactor create a business case. For example, if you are concerned about the traffic load to the website from a new campaign, you can use the max as a “worst-case scenario” traffic volume to split your deployment.
First, find the range of benchmarks, and then determine one that is most applicable for your situation. Here is what I have selected for the Non-Profit benchmarks.
The range is useful if you are trying to estimate the impactor create a business case. For example, if you are concerned about the traffic load to the website from a new campaign, you can use the max as a “worst-case scenario” traffic volume to split your deployment.
Figure 2. Finding the range of and the most applicable email benchmarks
The Most Applicable number is what you would use to gauge the performance of your own emails. If you added your own historical data to the list, it would likely be your choice here.
Note that we use the same study across multiple metrics to ensure an apples-to-apples comparison. It’s also much easier for your audience to follow when you are presenting your results.
In this case, we don’t have a historical. Here are some factors that led us to choose Constant Contact’s numbers as our benchmark.
Note that we use the same study across multiple metrics to ensure an apples-to-apples comparison. It’s also much easier for your audience to follow when you are presenting your results.
In this case, we don’t have a historical. Here are some factors that led us to choose Constant Contact’s numbers as our benchmark.
- Avoid studies with skewed results. Campaign Monitor’s benchmark shows a much lower open rate than the rest. This indicates the emails included in their study may not be representative of the industry.
- Know the bias. Klaviyo is an ESP mostly used by eCommerce brands. Therefore, its “All Industry” benchmarks would skew towards eCommerce-related emails. This explains the lower clickthrough rate compared with other benchmarks, and hence it’s not a candidate for “Most Applicable”. (In reality, knowing this bias, we would have excluded this in the shortlist from step 2.)
- Give preference to more specific results. Moststudies have both the “All Industry” average and non-profit-specific results.We would generally pick the non-profit-specific results over the generalaverage.
We made a subjective call to use the Constant Contact Nonprofit Services metrics as our Most Applicable benchmark. It is a close call with the GetResponse Non-Profit metric. One reason is that Constant Contact has a more US-centric customer base while GetResponse has a more global customer base. We might pick GetResponse if our client had a more global audience, or if we wanted to benchmark against a more ambitious clickthrough rate target. We can’t completely take out the subjectivity, but hopefully this is a more informed choice than using the first study that comes up on a Google search.
The range is useful if you are trying to estimate the impactor create a business case. For example, if you are concerned about the traffic load to the website from a new campaign, you can use the max as a “worst-case scenario” traffic volume to split your deployment.
The range is useful if you are trying to estimate the impactor create a business case. For example, if you are concerned about the traffic load to the website from a new campaign, you can use the max as a “worst-case scenario” traffic volume to split your deployment.
4. Refresh the list periodically
I recommend updating the list once a year, around February /March. Most annual wrap-ups will be published by this time, so you can get the latest data in one go.
At the same time, remove studies that are over 2-3 years old from your list to keep the results current and relevant.
If a report has changed its categorization, add in the new one if it’s relevant. I would sometimes carry the old one for an extra year if it is unique enough compared to the new one. For example, if a study has gone from having a bank category and an insurance category to just having a financial services category, it may be useful to keep all 3 for reference for a year. This is why we show the timing of each benchmark in our shortlist so we have a frame of reference.
At the same time, remove studies that are over 2-3 years old from your list to keep the results current and relevant.
If a report has changed its categorization, add in the new one if it’s relevant. I would sometimes carry the old one for an extra year if it is unique enough compared to the new one. For example, if a study has gone from having a bank category and an insurance category to just having a financial services category, it may be useful to keep all 3 for reference for a year. This is why we show the timing of each benchmark in our shortlist so we have a frame of reference.
Can I do this with an AI Assistant?
Some of you are probably asking: “Can I not just ask an AI Assistant to give me the benchmarks?”
We used 2 different AI Assistants – ChatGPT-4o and Claude3.5 Sonnet – to help us curate email benchmarks as in the example above. Both returned results from the MailerLite study, which had the highest open rate and clickthrough rate across all studies. In the case of Claude 3.5, it was notable to retrieve the clickthrough rate despite multiple different prompts.
We used 2 different AI Assistants – ChatGPT-4o and Claude3.5 Sonnet – to help us curate email benchmarks as in the example above. Both returned results from the MailerLite study, which had the highest open rate and clickthrough rate across all studies. In the case of Claude 3.5, it was notable to retrieve the clickthrough rate despite multiple different prompts.
Perhaps with additional fine-tuning, or with other assistants, we would be able to achieve similar results. But in the end, if you want to gauge your program’s performance against a benchmark, wouldn’t you want to know what you are comparing yourself against?
We hope you’ve learned some nuggets from this benchmarking how-to. Drop us a line on how you use benchmarking to help you improve your marketing.