Marketing metrics you shouldn’t take at face value, and why
Measuring data can be just as important as ignoring data. We cover different metrics that you should pay extra attention to before committing to "data-driven" decisions.
There’s much discourse on “socials” on how “being data-driven is out” because marketing data is often wrong. It’s not (necessarily) that the data is wrong, it’s that people making decisions based on that data often don’t bother understanding how that data is calculated in the first place.
Without the correct context, you do, indeed, risk making some questionable business decisions. Just because a report says “Unique Users” or “Direct Concversions”, it doesn’t actually mean you’re getting that.
In this article, we cover misconceptions about what some common metrics represent and how to best leverage marketing data.
All marketing data are estimates
First things first: marketing data is not that maths, it’s analytics. It’s interpretations. It’s unlike finance, where you have an X number of sales of Y value. The numbers you see on marketing reports are not “real”. Afterall, how can any tool or service be sure that a campaign was responsible for convincing someone to purchase a product? Spoiler alert: they can’t.
Obviously, that doesn’t mean you shouldn’t use marketing data. Leveraged wisely, it is a powerful competitive advantage that can help you find and acquire valuable users cheaply.
But to make decisions based on marketing data, you need to be aware of how it’s measured and how it can be wrong. As one of my favorite quotes says:
Metrics you should take with a grain of salt, and why
Google Ads conversions
Would a user have converted anyway (via organic search) if they hadn’t clicked on your paid search impression?
The very nature of Paid Search implies cannibalization. The most obvious example of cannibalization is brand search because most companies rank a top page for their brand search.
The more successful you are at SEO, the bigger a concern paid search cannibalization becomes.
Just because Google Ads says you have 100 conversions, it doesn’t mean that these are incremental conversions. That also doesn’t mean you should put all your spend on brand search.
Actually, make that all ad platform conversions
Although Paid Search is the most common example, the cannibalization effect is not just restricted to Google Ads. Retargeting audiences (running on social, YouTube, programmatic, etc.) all have cannibalization problems. Perhaps these strategies have “snagged” the conversion, but were they really responsible for them in the first place?
The issue with ad platform conversions is not just cannibalization. Sometimes, the problem is the exact opposite.
Certain strategies “underattribute”, due to cross-device journeys, long lookback windows or lack of clicks.
Estimated search volumes from ahrefs, SEMrush, etc
From my experience, these are so often very very wrong. I’ve had multiple clients where search volume for their intent keywords were deemed very low with sometimes literally zero predicted searches per month.
However, when we start running paid search for those keywords, we see there’s indeed existing demand for them. Sometimes even thousands of monthly searches.
The only way you can really know the search volume of a keyword is to bid for it on Paid Search.
Unique users, anywhere but your own
Tools–-from ad platforms to website analytics like Google Analytics-–use different signals to identify if a user is indeed “unique”. Common leveraged parameters are: device identifiers, cookies, browser headings, IPs, location.
But, as you can suspect, none of these things guarantee a user is unique. I have multiple devices. My IP has changed. My cookies have expired. I access websites from different browsers.
The real number of unique users is always less than what’s reported.
The real number of unique users is always less than what’s reported.
Unique users, even your own first-party data
Ever relying on a product identifier, like your very own user_id doesn’t guarantee an actual measurement of unique users. That’s because certain products require users to create multiple accounts. (You can read more about first party data entitles here.)
For example, every time I get a new invitation to a client’s Slack or join a new community, I have to create a new account.
There are also products where you don’t need to create multiple accounts to use, but people still do. Teens have “ristas” and “instas”: several Instagram accounts for unique use cases. Multiple X accounts are also common.
Facebook has previously gotten schtick for counting these “multiple accounts” as multiple users in their shared MAU reports. And we can’t forget Elon’s statement that Twitter’s MAU was overinflated because of bots.
Average time on site
This is a problem of using “averages” in general. Depending on the size of your data set, it can be very easy for a single user to affect your “Average time on site” metric. If they leave the tab open for a long time, historically, that has counted as time on site.
If you want to measure time on site as an engagement metric (fair enough, I say) I would suggest either:
Using BigQuery to determine sessions and session length
Creating a “timer event” that fires once the user has spent an X amount of time on site, for example, 15 seconds. I’ve used this a lot as the primary optimization event for brand awareness campaigns
How to best leverage marketing data
Understand edge cases from each metric
As we’ve seen, there are several use cases where the metric doesn’t really mean what it means. Different metrics have different edge cases.
The best way to leverage marketing data is to understand how each metric is calculated. How are conversions from a certain platform attributed? How are sessions defined? How are users counted?
The best way to leverage marketing data is to understand how each metric is calculated. How are conversions from a certain platform attributed? How are sessions defined? How are users counted?
There’s no short path here, just an ongoing learning process on the “whys” of different data.
This understanding is beneficial not just for data interpretation, but also for identifying when vendors in the data space are overselling what they can deliver. RFPs become a lot smoother when you know what’s being promised is technically impossible.
Track across multiple sources
It’s also helpful to track comparable metrics across different sources. Discrepancies can tell you important stories you were previously unaware of and highlight how essential it is to understand how numbers are calculated.
Test, test, test
Marketing metrics are not very useful if they live in silos. Analytics is better supported with hypotheses and tests.
A/B tests are great, and control groups are even better. For paid media, holdout tests can deliver the best overview of what’s working than any marketing tool can deliver.
Do you trust the data you use to report on campaigns?
If you’re struggling to make sense of what metrics to use or what data sources to rely on, please do reach out! I’m a consultant who helps companies drive marketing results with the help of attribution, analysis, education, and more.
Book a free 15-minute consultation call to see if I can help: