We recently had a conversation with Google’s Large Client Team about one of our top-performing accounts. Their advice was direct: the single most impactful tactic the client could implement was to switch from Standard Shopping to Performance Max.
They were so confident that they helped design a full A/B experiment to prove PMax was the superior campaign type. So we ran it. We followed their setup, tracked the data meticulously, and after several weeks, we got a result that even surprised me. PMax lost.
In this article, I’ll walk you through the experiment setup, what the data actually said, and the key takeaways I have from seeing PMax fail to beat a simplified version of our Shopping strategy.
Go beyond the article
Why the video is better:
- See real examples from actual accounts
- Get deeper insights that can’t be conveyed in writing
- Learn advanced strategies for complex situations
The Experiment Setup: Biased in PMax’s Favor
To be clear, this was a proper experiment. The setup was verified by the Large Client Team at Google. We ran it in a top-performing account and had dedicated Search campaigns running alongside to isolate the variables. We also disregarded the first week of data to let both campaigns exit their learning phases.
But there’s a critical piece of context here. To run a one-to-one experiment, we had to severely handicap our Standard Shopping setup.
Specifically, we couldn’t use:
- A complex Shopping structure with campaign priorities.
- Shared Budgets across campaigns.
- Portfolio Bid Strategies.
These three elements are core to how we extract maximum performance from Standard Shopping at Savvy. By removing them, we were essentially fighting with one hand tied behind our back. So the test wasn’t PMax vs. our best Standard Shopping setup. It was PMax vs. a basic, stripped-down version. This is an important point to remember.
The Data: What a Lack of a Clear Winner Really Means
Over the eight weeks we ran the test, the winner went back and forth. The final result wasn’t statistically significant according to the 95% confidence interval I wanted. But the lack of a clear winner is a conclusion in itself.
The entire premise from Google was that PMax would outperform Standard Shopping by a “clear margin.” When it failed to do that even against a handicapped setup, the point was made. We could have run the test longer, but with performance being so close, neither the client, Google, nor I saw the point in continuing.
The best data point is the cumulative profit from each campaign. Standard Shopping is in green.
From day one, our simplified Standard Shopping campaign generated more profit. With the exception of just six days during the 42-day experiment, it stayed ahead, and the profit gap widened as time went on.
The Problem with Volatility
One of the reasons PMax generated less profit was its volatility. It had several negative-profit days. Our Standard Shopping campaign didn’t have a single one. While PMax also had some higher-profit days, its instability ultimately resulted in less overall profit compared to the consistent performance of Standard Shopping.
The common counter-argument is that six weeks isn’t enough time for PMax to settle down. But this wasn’t an isolated case. We ran PMax in two other accounts for nearly nine months (38 weeks), and the performance never improved beyond the initial learning phase. Those accounts showed similar volatility over a much longer timeframe.
The conclusion across all three accounts was the same: PMax was not an automatic upgrade over a well-managed Standard Shopping structure.
Channel Allocation: Where Did The Money Actually Go?
I was genuinely surprised by the channel allocation data. We provided PMax with fully optimized assets for Display and YouTube, hoping to see it expand into new channels. It didn’t.
The campaign was almost entirely Shopping. The tiny amount of spend we saw on Display and YouTube was not successful by any metric (and the Display spend was likely just retargeting). This was especially strange because the POAS target was set well below our account average, so there should have been plenty of room for PMax to experiment.
If the campaign is still 95%+ Shopping, why would I give up control over negatives, structure, and budgets unless I get a clear performance gain? In this case, we didn’t.
This was even more apparent in the two other accounts that ran for much longer. In those accounts, we didn’t run any dedicated Search campaigns, giving PMax a clear opportunity to fill the gap. It still only managed to allocate a meager 4-13% of its budget to Search. Our old DSA campaigns performed better than that.
My 5 Key Takeaways from the PMax Experiment
If I had to summarize the entire experiment, it boils down to five key takeaways that are applicable to most e-commerce businesses.
1. The test was rigged for PMax to win
I can’t stress this enough. PMax didn’t fail to beat our best, most sophisticated Standard Shopping setup. It failed to clearly beat a simplified version that we would never run for a client in a real-world scenario.
2. PMax was mostly a Shopping campaign anyway
Even with optimized assets and bidding flexibility, PMax didn’t unlock new growth on YouTube or Display. It was just a black-box version of a Shopping campaign. This begs the question: what is the actual upside if it’s not expanding your reach into new channels effectively?
3. Strong Search campaigns limit PMax’s upside
PMax isn’t magic. It needs to find incremental value somewhere. In this account, we already had strong, dedicated Search campaigns covering our core terms. With Shopping already optimized and Search already covered, where was PMax supposed to generate a significant lift?
4. Stability has value
The averages were close, but the day-to-day reality wasn’t. Standard Shopping was stable and predictable. PMax was volatile, with high peaks and low valleys (including negative-profit days). When you are actively managing and scaling an account, stability is a valuable asset. It makes decision-making much easier.
5. “Try PMax” is not a strategy
This is my biggest takeaway. “Oh, you’re running Standard Shopping? You should try PMax” has become the laziest recommendation in digital marketing. A campaign type is not a strategy. You have to look at the underlying engine, the channels it serves, and the levers you lose access to. If that doesn’t match your business goals, you’re just throwing pasta at the wall and hoping it sticks.
Conclusion: PMax is a Tool, Not a Magic Pill
My conclusion from this experiment is not that PMax is bad. It’s a powerful tool in the right situation.
My conclusion is that PMax should not be treated as an automatic, one-size-fits-all upgrade from Standard Shopping. It’s a trade-off. You trade control for automation. And as this test showed, that trade-off doesn’t always result in better performance. Sometimes, a well-built, strategically managed Standard Shopping setup is still the better choice.
Like any tool, it only works when the job actually fits.
[TL;DR]
- The experiment was biased in PMax’s favor, as it competed against a handicapped Standard Shopping setup without our key strategic levers.
- PMax failed to generate meaningful spend or performance on YouTube and Display, acting mostly as a less transparent Shopping campaign.
- The presence of strong, existing Search campaigns significantly reduces the potential for PMax to add incremental value.
- PMax was more volatile than Standard Shopping, leading to more negative-profit days and less consistent overall performance.
- Choosing a campaign type is a strategic decision, not a default action. “Just try PMax” is lazy advice that ignores an account’s specific needs and strengths.
