The Single, Simple, All-Encompassing Reason You Should Test Your Ads

It’s actually no more complicated than this: You should test your ads because you don’t know what will and won’t work. No matter your level of expertise, the refinement of your judgment and taste, the decades of experience or rarity of your sheer marketing talent, you literally can not predict advertising performance. You might really, really believe you can. But that doesn’t change the fact that you can’t.

Originally published

September 29, 2020

Written by

It’s actually no more complicated than this: You should test your ads because you don’t know what will and won’t work. No matter your level of expertise, the refinement of your judgment and taste, the decades of experience or rarity of your sheer marketing talent, you literally can not predict advertising performance. You might really, really believe you can. But that doesn’t change the fact that you can’t.

Don’t believe me? (Even if you do, just play along, it will be fun.) Let’s try it! Have a look at the ads in the group below and pick the one you think generated the best result.

Can you guess the winner?

The first ad offers gorgeous, appetite-teasing closeup detail. Yum! The second gives you a visual pop with the saturated background to stand out in the feed while showing optionality, like a party with a whole table of treats. The third switches up the post copy for a punchy bit of fun, and it lets you project yourself into the action with a captured moment our four-year-old inner selves can all relate to: Crowding around the birthday kid, waiting to be handed our slice, anticipating that first bite bursting on our taste buds like a giant, sugary confetti popper of pure flavor! Yes!

Anyway, about those ads. Do you have a prediction?

The winner is the one on the left, with a cost per acquisition (CPA) almost 25% lower than the one in the middle and a smidge more than 14% improved from the one on the right. That’s a massive difference in performance.

Perhaps you knew that already, though. Maybe you guessed right. Here’s the thing: You were just lucky. Let’s try it again, with a new set of ads.

None of these things is quite like the others.

Here you see three different images in five ads. But aren’t 1 and 2, and 3 and 4, the same? Look closer and you’ll see the subtle difference between the ads. Savvy digitals will appreciate what’s going on here. We’ve taken the winning approach from our first test and generated more variations on it—having learned that the close-up worked the best of our first three ads, we’re now trying to figure out how to do the best close-up ad possible. Do you have a guess which generated the best result?

It’s number 3, with a CPA 33% better than number 1, the worst performer of the group. So the meme-like text overlay was an improvement over the simple close-up. But there’s more going on here that’s worth pulling apart. Number 2 significantly outperformed number 1, and 3 significantly outperformed 4—the only difference in both those cases was the tiny addition of an itty-bitty little birthday cake emoji waaaaay down at the bottom of the ad.

Details really do matter, and the more you scale up your budget, having identified an ad generating a good result, the more the details matter. In dollars. In revenue. In lifeblood.

The lesson: Subtle changes to the creative generate material differences in business outcomes.

Did you guess right again, and predict not only that the meme-style overlay AND the addition of the birthday cake emoji would produce the best result? That’s pretty incredible! Let’s try one more set of ads and see if you can pick the winner.

How about these?

Here you see a different approach to the creative, showing flavor options and the confetti-toss signaling the moment of celebration. See if you can make a prediction. Ready?

The winner is number 1, with a CPA almost 32% better than the next best performer, number 2, and 89% better than number 6, the worst performer in the group. If you think about the impact of spending your ad budget on number 1 instead of on 6, a comparative dud, that’s an absolutely huge difference in return on ad spend (ROAS). Armed with the understanding of that difference in results, you could make choices with a meaningfully positive impact on your business. Amazing.

Ok, you’re right. This was sort of a trick. You’re not crazy: The ads in this last group are identical. The difference is not in the creative but in how the ads were targeted. In this case the audience reached by the first ad was more responsive than the audiences reached by the others.

The point here is that testing your ads involves more, potentially much more, than simply swapping out images and writing alternate versions of your copy. Many factors, from creative to audience to ad unit choice to bidding strategy to time of day, geography, and more have the potential to impact the results your ads generate.

Not only that, but the results you can generate by testing your way into better ads can and will change over time. What works today might not work tomorrow. Best in class marketers (like the team at Milk Bar) continuously test numerous variations of all the factors that impact their results, building a strong discipline of always learning, always experimenting, always finding new ways to improve.

It might sound overwhelmingly complicated, and you can certainly make it so with attribution modeling, automated ad production, holdout groups, and rooms full of PhD’s nightmarishly arguing with each other about statistical certainty and confidence intervals, but it truly doesn’t need to be like that. Start simple, with basic tests like the one in the first example above.

If you’re testing at all, even with two versions of a simple tagline, you are likely already generating material benefit to your business. And if you aren’t testing your ads, don’t beat yourself up.

But do ask yourself: Knowing now that you might not actually know what you thought you knew, do you really want to keep relying on random predictions to decide what good looks like in advertising, which ads to run, how much to spend, or which of your infinite options are going to generate positive results for your business?

I’m guessing not.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

, last updated on

September 29, 2020

No items found.

Let's do this!

Get our newsletter
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.