Data Discrepancies in Marketing Sources: what they are, why they happen and what to do about them

Discrepancies between marketing data sources undermine trust in the data and lead to multiple problems. How important is it for data sources to match and how should we do this?

Data is so pivotal for marketing that you’re most likely tracking digital campaigns with multiple tools. 

For example, it’s commonplace for the conversion event for a Facebook campaign to be measured on the ad platform (Meta), the ad server (Campaign Manager), the marketing analytics (Google Analytics) and first party data tool (Snowplow).

Most likely, none of the conversion events attributed to that campaign will match between the four tools. 

Why do they not match? How do we fix this? Should we fix this? 

In this article, I will explain common marketing sources along with some of their associated discrepancies, explore why (some) of the discrepancies are problematic and propose solutions to these challenges.

The Three Challenges Posed by Data Discrepancies

Data discrepancies are to be expected, since different sources capture, interpret and model data in different ways. But, if not discrepancies are not addressed, they cause several issues for organisations:

1. Distrust in the Data → Media Misinvestment

Inaccurate or conflicting data can make it difficult to determine which source is reliable, resulting in hesitancy to use the data for critical business decisions. 

This is especially severe when organisations double down on “one source of truth”, because each data source has its own blind spots.

For example: a marketing team that only trusts Snowplow (post-click, cookie-based) data and ignores the signals from the Ad Platform (post-view, user-based). 

This can lead to the team to miscalculate the true impact of the media, because conversions may be incorrectly attributed to direct traffic. The knee jerk reaction is investing in media that is easier to measure, but could be less effective. I’m looking at you, Brand Search Ads.

2. Misalignment → Conflicts

Discrepancies can create confusion between external agencies and internal teams, as they may use different data sources. 

For example: a marketing agency that only has access to Campaign Manager data, while the in-house team only looks at Segment-powered Tableau dashboards.

This miscommunication can hinder the development of a cohesive strategy. Education and clear communication across departments are essential for addressing this challenge.

3. Unnecessary Investigation → Time Wasted

This one is a consequence of both data distrust and team misalignment.

Time-consuming and often frustrating investigations into discrepancies can waste valuable resources, especially when the discrepancies are not significant and do not impact decision-making.

These investigations can suck in multiple internal departments (data engineering, data analytics, paid media, marketing analytics) and external stakeholders (marketing agencies, analytics agencies).

To tackle data discrepancies, marketers should accept that a single source of truth will never have the full picture. Instead, leverage multiple sources to create a comprehensive understanding of your data landscape.

Explaining Discrepancies in Common Marketing Sources

The reasons for discrepancies are aplenty. Below, you can find some common marketing sources accompanies by their associated discrepancies:

Google Analytics UI

• It attributes conversions to all campaigns (UTMs and linked ad accounts) that you’ve defined, but also to standard sources (e.g. “default channel grouping”) such as direct and organic. 

• It uses Google User ID instead of cookies to identify users across devices and sessions. (Lasts longer.)

• Comes with an out of box attribution model with minimal visibility of the marketing touches.

• Only tracks post-click conversions. (Once the user has entered your analytics property.)

First-Party / First-Party Server Side (e.g. Snowplow, Segment)

• Much more custom. It’s dependent on how you identify anonymous (non-logged in) users across sessions and devices. E.g. fingerprinting, cookies, etc. 

• And also dependent on the marketing campaigns you’re tracking and the attribution model you’ve defined.

• Like Google Analytics, only tracks post-click conversions. (Once the user has entered your analytics property)

Ad Platform (Meta, Google Ads, etc)

• Only attributes conversions to the ad platform itself. On Meta, it doesn’t matter if a user has clicked on 10 Search Ads after being served a Meta ad. If they convert, then they claim the conversion.

• Dependent on the lookback window and attribution model you’ve picked.

• Because it tracks the user before they enter your website, also tracks post-view conversions.

Ad Server (Campaign Manager, Kevel, etc)

• Like the Ad Platform, because an ad server can track the user before they enter your website (measuring impressions), it can also track post-view conversions.

• Also dependent on the lookback window and attribution model you’ve picked.

• It can attribute conversions to multiple ad platforms, but, unlike Google Analytics or First-Party, it can’t attribute to Organic / Direct.

Four Solutions for Marketers Facing Data Discrepancies

When dealing with discrepancies in data from different marketing sources, marketers should follow these best practices to ensure they make informed decisions based on accurate information:

1. Establish acceptable limits of discrepancy

Understand that discrepancies will occur, and determine what range of discrepancy is acceptable for your business. This threshold will vary depending on the platforms, campaigns, and products involved.

Inform different stakeholders of the reasons behind these discrepancies to avoid unnecessary investigations.

2. Monitor discrepancies

Implement a system to monitor discrepancies, either through automated tools or regular manual checks. This will help you stay on top of any significant changes that may require further investigation.

In the next session, we cover some automated tools that can be built with this goal in mind.

3. Understand the nature of the campaign, platform, and product

Recognise that different campaigns and platforms may have varying levels of discrepancy. For example, TikTok campaigns might show higher discrepancies when compared to Google Search campaigns. This could be because of cross-device tracking (TikTok will serve on mobile) or the time-to-conversion (Google Search can reach leads with high intent). 

4. Expand with zero party data and econometrics

“Zero party data” such as attribution surveys (asking new users where they found out about your product) can offer a fuller picture of hard to attribute sources. 

I’ve seen this clearly when working on a mobile app. Influencer marketing—who was getting nearly zero attributed installs monthly—was responsible for 40% new users in the attribution survey.

Econometrics studies are also gaining traction as a way to measure short term ROI and long term ROI across different marketing channels, including OOH and brand campaigns.

Real-World Solution: Discrepancy Dashboard and Slack Bot

I’ll cover two solutions I’ve helped a marketing agency implement to monitor discrepancies for their clients. These don’t fix discrepancies, but have helped the team identify when to investigate discrepancies.

Discrepancy Dashboard

The dashboard joined the data sources they used: their DSP (DV360), ad server (Campaign Manager) and brand safety tool (IAS). It reported on impressions, based on each marketing source.

The DSP reported on all impressions that were purchased. The ad server reported on all impressions that were purchased and served. The brand safety tool reported on all impressions that were purchased, served and displayed (deemed valid).

As a result, the number of impressions decreased with each level, leading to discrepancies among these sources.

Discrepancy Slack Bot

Based on historical data, we determined what were the baselines discrepancies among these marketing sources.

We then built a Slack bot that would notify programmatic traders when a placement’s discrepancy surpassed a specified threshold, such as 10%. 

This alert system helped the client identify potential fraud, ads placed in the wrong locations, or tracking issues. It was essential to investigate and resolve these problems promptly.

How I Learned to Stop Worrying and Love Discrepancies

Discrepancies in data from marketing sources can cause distrust in the data, miscommunication between external agencies and internal teams, and wasted time investigating perfectly reasonable discrepancies. 

To address these challenges, marketers must understand the differences between sources, report on multiple conversion sources, and be aware of the discrepancies in the marketing sources they use.

By embracing best practices and being proactive in addressing discrepancies, marketers can make more informed decisions and enhance their marketing strategies. Remember, the key is not necessarily to find a perfect one-to-one match, but to exercise common sense, maintain open communication, and establish guardrails to make the most of your marketing data.