Measurement: Less is More

Why statistically significant results are the wrong goal for many marketing teams.

Originally published

September 28, 2020

Written by

Jacob TallyJacob Tally

Product Marketing & Operations

A/B testing is the best way for marketers to figure out what works. So why don’t more marketers do it? Likely because the term “a/b testing” carries a lot of baggage. It’s perceived as a world inhabited by data scientists and performance marketing gurus, roads and cities laid out like excel spreadsheets, and governed by a statistically significant code of laws. Frankly, it sounds intimidating.

However, in this case, perception is not reality. In practice, the “right” way to test is the way that produces actionable learnings as quickly as possible, something that will vary across marketing orgs. Most of the testing “how tos” you’ll find are written by marketers (or data scientists) at larger, established marketing orgs or from testing solutions providers speaking to that same audience. But the marketers who most need “how to” guidance and who could benefit from establishing a habit of testing and learning operate in a very different environment — one in which they wear multiple hats, juggle different roles, and have to move at an entirely different (faster!) pace. What works for one doesn’t work for the other, and trying to use the same tactics in each environment can even be detrimental.

Smaller marketing teams and startups have different needs and constraints than those found at larger companies. Not only are time, resources, and overall data and conversions fewer, the types of variables/themes/concepts each needs to test are going to be very different. Testing to find your way to product market fit is going to require different tactics and strategies than testing the specific elements of an established product’s product-details-page (PDP).

Ultimately, for marketing teams at small to medium-sized businesses (SMB’s), testing more, testing faster, and not being beholden to statistically significant results is going to be the fastest path to success. That means measuring fewer things while focusing only on the most important metrics. This gap - the difference between what SMB marketers need and what’s often espoused as “best practice” - creates an opportunity. Smaller marketing teams can benefit greatly from solidifying the marketing fundamentals within their organizations, but they need that process to be simplified and supported with easy to use tools. They need:

At Jove, we’ve worked with clients and have proven the success of this process — a process whose parts form the fundamentals all marketers should rely on. And now, for the first time, we’re excited to bring to market a collection of tools that make this process feel simple.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

, last updated on

September 28, 2020

Hello world.

A new company making fabulous advertising tools for small and remote marketing teams. Read on to see what we're all about and understand our unique point of view on testing ads.

Let's do this!

Get our newsletter
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.