Chapter 09: The Main Challenges Facing the AdTech & Programmatic Advertising Industry

The evolution of programmatic advertising is a story of remarkable innovation. In just a few decades, the industry has built a global, automated, and highly efficient ecosystem for buying and selling digital media. However, this rapid growth has also created a host of complex and persistent challenges.

In this chapter, we will confront the challenges plaguing the industry head-on.

We will explore the issues of transparency, fraud, privacy, and brand safety that every player in the supply chain must navigate.

We will also examine the structural threats posed by walled gardens and the challenges associated with managing the rise of artificial intelligence.

Key takeaways

• From the opaque “AdTech tax” that consumes a significant portion of advertiser budgets to the constant battle against sophisticated ad fraud, the programmatic landscape is fraught with difficulties.
• The foundational pillars of the ecosystem are being reshaped by the seismic shifts in data privacy, the growing dominance of walled gardens, and the rise of AI.
• The lack of fee transparency is caused by complex, multi-hop supply chains that make it difficult for advertisers and publishers to see how media spend is distributed and how much reaches working media.
• Ad fraud costs the industry approximately US$84 billion each year with sophisticated invalid traffic and spoofing techniques used to divert spend and distort performance across open programmatic environments.
• Privacy regulation and signal loss (e.g. cookies, mobile IDs) have fragmented identity, reducing addressability, measurement accuracy, and cross-channel consistency.
• Ensuring ads appear in appropriate, high-quality, and brand-safe environments remains difficult at scale, particularly across diverse and user-generated content.
• Dominant walled-garden platforms like Google, Meta, and Amazon concentrate data, inventory, and measurement within closed ecosystems, limiting transparency and interoperability.
• The rise of AI is reshaping campaign execution and optimisation but also introducing new challenges around transparency, control, and copyright.

A lack of transparency into AdTech fees

One of the longest-standing criticisms of the programmatic ecosystem is its inherent lack of transparency.

For an industry built on data and precision, it can be surprisingly difficult for advertisers to get a clear answer to two fundamental questions: “Where did my money go?” and “Who really saw my ads?”
This opacity manifests primarily in the supply chain’s cost structure.

The journey from an advertiser’s budget to a publisher’s revenue involves numerous intermediaries – agencies, DSPs, ad exchanges, SSPs, and verification vendors – each taking a fee.
The portion of spend consumed by these intermediaries is often referred to as the “AdTech tax.”

A landmark 2020 study by the UK’s advertiser trade body, ISBA, provided a stark illustration of this issue.

The report found that, on average, publishers received only 51% of the money advertisers spent.

Even more concerning was the discovery of an “unknown delta,” where 15% of the advertiser’s budget could not be attributed to any known participant in the supply chain, effectively vanishing into the system’s complexity.

The diagram above illustrates where an advertiser’s ad spend goes to different parts of the media supply chain as a percentage.

Source: ISBA Programmatic Supply Chain Transparency Study, 2020

This lack of financial transparency makes it difficult for advertisers to assess the true value of their media spend and for publishers to understand the true market value of their inventory.

One of the main criticisms of the fee transparency issue in AdTech isn’t the fees themselves, it’s the lack of transparency into them and the difference between what is reported and what is actually taken.

Over the years, many companies – from publishers to advertisers and agencies – have tried to find out the answers themselves.

One case from 2017 involved The Guardian newspaper suing its SSP partner, Rubicon Project (now Magnite), claiming that the SSP had taken “secret commissions” in addition to its disclosed fixed-percentage fees.

More recently, the push for fee transparency has come from advertisers and ad agencies, setting their sights once again on SSPs and the supply side.

The aim of their crusade is to ensure that supply-side vendors (e.g. SSPs) take only their disclosed fees, as the more ad spend that reaches publishers, the greater their chances of winning impressions.

But due to the opaque nature of the media supply chain, the topic of fee transparency is set to be one that plagues the industry for years to come.

Ad fraud

Ad fraud is the practice of deliberately generating invalid advertising interactions to illegitimately siphon money from advertisers’ budgets.

It is a problem that costs the industry approximately US$84 billion each year. In addition to the financial losses, ad fraud undermines trust in the digital advertising ecosystem and wastes significant marketing investment.

Fraudsters employ a wide range of ever-evolving techniques to deceive advertisers and publishers alike.

The industry broadly categorises fraudulent activity as invalid traffic (IVT), which is any traffic not generated by a real human with genuine interest.

IVT is further broken down into:

General invalid traffic (GIVT): This includes traffic from known data centre IP addresses, search engine crawlers, and other legitimate non-human sources that can be easily identified and filtered.

Sophisticated invalid traffic (SIVT): This is much harder to detect and includes traffic generated by hijacked devices, ad-viewing bots designed to mimic human behaviour, and other malicious activities.

Common ad fraud schemes include:

Domain spoofing: A fraudster misrepresents a low-quality website (e.g., a site filled with bot traffic) as a premium, high-value publisher in the bid request. Advertisers believe they are bidding for an ad on a reputable news site and pay a high CPM, but the ad is actually served on a worthless site. The IAB’s ads.txt initiative was created specifically to combat this problem.

Ad stacking and pixel stuffing: These techniques involve loading multiple ads in a single ad slot, either by layering them on top of each other (stacking) or by stuffing them into a 1×1 pixel (stuffing).

While only the top ad is visible, impressions are registered for all the ads in the stack, defrauding advertisers.

Click farms: This is a more manual form of fraud where large groups of low-paid workers are hired to repeatedly click on ads to generate fraudulent clicks and deplete competitor budgets.

Privacy and fragmented identity

As we explored in detail in Chapter 08, data privacy has moved from a niche concern to a central organising principle of the digital world.

The combination of comprehensive regulations like the GDPR, state-level laws like the CCPA, and platform policy changes from Apple and Google has fundamentally fragmented the landscape for user identification and measurement.

This fragmentation is one of the most significant challenges facing the industry today.
The decline of the third-party cookie and the limitations on mobile advertising IDs mean there is no longer a single, consistent way to recognise and interact with users across the open web.

This directly impacts core advertising functions:

• Targeting: Behavioural targeting and retargeting at scale are much more difficult without a persistent cross-site identifier.
• Measurement and attribution: As detailed in Chapter 07, connecting ad exposure to conversions across different domains and devices is a major challenge, undermining the industry’s ability to prove ROI.
• Frequency capping: Without a way to recognise a user across different publisher sites, it’s difficult to control how many times they see the same ad, leading to poor user experiences.

The industry is actively working to address this challenge through the collection of privacy-safe alternatives, such as universal IDs and the use of data clean rooms.

However, as there is no one-to-one replacement for third-party cookies and mobile IDs, the transition is complex and ongoing.

Brand safety

Brand safety is the process of protecting a brand’s reputation by ensuring its advertisements do not appear alongside content that is illegal, harmful, or otherwise inappropriate.

In the high-speed, automated world of programmatic advertising, where ads are placed across millions of websites in milliseconds, the risk of an ad appearing next to objectionable content is a major concern for advertisers.

It is important to distinguish brand safety from the related concept of brand suitability.

Brand safety vs. brand suitability

Brand safety is about avoiding universally toxic content, such as hate speech, terrorism, illegal activities, or extreme violence. This is a universal requirement for almost all advertisers.

Brand suitability, on the other hand, is about aligning ad placements with a specific brand’s values. The content may not be unsafe, but it might be unsuitable for a particular brand. For example, an airline might not want its ads to appear next to a news article about a plane crash, or a children’s toy brand might want to avoid content related to political conflict.

Advertisers manage brand safety and suitability by using services from third-party verification companies like DoubleVerify and Integral Ad Science.

These platforms use advanced contextual intelligence to analyse the content of web pages in real-time, blocking ad placements on pages that violate a brand’s predefined safety or suitability rules.

The rise of walled gardens

As we have referenced throughout this book, walled gardens are closed digital ecosystems where a single company controls all the data and advertising technology.

The most powerful walled gardens are operated by Google, Meta (Facebook and Instagram), and Amazon.

While these platforms offer advertisers immense reach and highly effective targeting capabilities based on their rich first-party data, their dominance presents several structural challenges to the broader advertising industry.

Data silos

Walled gardens do not allow advertisers to export user-level data. This means an advertiser can see how their campaigns performed within Facebook or on YouTube, but they cannot easily connect that data to user interactions on other platforms or on their own website.

This makes true cross-channel measurement and attribution, as we discussed in Chapter 07, nearly impossible.

A lack of transparency and independent verification

Advertisers are often forced to use the platform’s own tools to measure performance, with limited ability to use their preferred third-party verification vendors.

This “grading your own homework” approach has been a long-standing source of friction.

Market dominance

The immense scale of the walled gardens gives them significant market power, leading to concerns about anti-competitive behaviour, rising ad costs, and a lack of negotiating power for advertisers.
As a result, these companies are facing increasing antitrust scrutiny from regulators around the world.

Artificial intelligence

As we outlined in Chapter 05, the introduction and rise of AI is one of the fastest growing trends ever seen not only in the AdTech and programmatic advertising industry but in the wider technology industry.

Artificial intelligence has permeated virtually every aspect of programmatic advertising, fundamentally reshaping how digital advertising operates at scale.

While most of the talk around AI is about its use cases and applications in AdTech and programmatic advertising, there are many more areas to this topic.

Use cases

AI has transformed AdTech by automating key functions such as real-time bidding, audience segmentation, dynamic creative optimisation, fraud detection, and performance attribution.
These technologies enable more precise targeting, tailored content, and better budget allocation across channels.

Beyond technology, AI is reshaping the workforce.
While some traditional roles—like manual media buying—are being phased out, new roles are emerging that blend marketing strategy with AI expertise.

AI-powered media planners now guide strategy and oversee AI systems, while creatives collaborate with AI tools. The shift highlights a growing demand for hybrid talent that bridges technical know-how and business impact.

An example of how AI is being used in media planning.

Source: The AI Media Planner by Spyrosoft AdTech

Skepticism

Despite AI’s widespread adoption, skepticism and trust issues persist throughout the programmatic advertising ecosystem.

Many advertisers remain uncertain about the “black box” nature of AI algorithms, struggling to understand why certain bidding or targeting decisions are made.

This opacity has created tension between the desire for performance improvements and the need for campaign transparency and control. Brand safety concerns have intensified as AI systems occasionally place ads alongside inappropriate content despite sophisticated filtering mechanisms.

The complexity of AI-driven attribution models has also generated skepticism, with some advertisers questioning whether these systems truly capture causation or merely correlation.

Trust issues extend to vendor relationships, where agencies and advertisers worry about over-reliance on proprietary AI systems that may not be fully auditable or explainable.

Overhype and backlash

The AI revolution in AdTech has not been immune to the broader patterns of technological overhype followed by backlash.

Early promises of AI completely revolutionising advertising effectiveness and eliminating wasted spend proved overly optimistic, leading to disappointment when reality fell short of inflated expectations.

The industry experienced a backlash against “AI washing”—vendors rebranding existing algorithmic capabilities as AI without meaningful innovation.

High-profile failures, such as AI systems learning biased behaviours or optimising for metrics that didn’t translate to business value, contributed to a more measured approach to AI adoption.

The programmatic advertising industry has gradually moved from blind enthusiasm to more pragmatic implementation, focusing on specific, measurable use cases rather than transformational promises.

This maturation has led to more realistic expectations and better integration of AI as a tool rather than a panacea.

Copyright issues

Copyright concerns have emerged as an increasingly complex challenge as AI becomes more sophisticated in creative generation and content analysis.

The use of AI to generate ad creative raises questions about the ownership and originality of machine-generated content, particularly when AI systems are trained on existing copyrighted materials.

Dynamic creative optimisation systems that automatically modify or combine creative elements must navigate potential copyright violations when adapting advertiser assets.

The deployment of AI for contextual advertising through content analysis has raised concerns about fair use, especially when algorithms process and analyse copyrighted content to make targeting decisions.

As generative AI becomes more prevalent in advertising creative production, the industry faces ongoing uncertainty about liability for copyright infringement and the need for new licensing frameworks that account for AI’s role in content creation and modification.