When Content Is Goliath and Algorithm Is David: What This New Research Reveals About Generative Search Optimization
A new research paper from the University of Washington has uncovered something counterintuitive about how generative search engines select content for citations. The findings challenge what many marketers assume about AI search optimization and reveal a surprising opportunity for brands willing to adapt their content strategy.
The paper, titled “When Content Is Goliath and Algorithm Is David: The Style and Semantic Effects of Generative Search Engine,” comes from researchers Lijia Ma, Juan Qin, Xingchen Xu, and Yong Tan. They analyzed approximately 10,000 websites across Google’s AI Overview and conventional search to understand what makes generative engines cite certain content over others.
What Makes Generative Engines Choose Your Content?
The researchers identified two mechanisms that drive citation decisions in AI-generated search results.
Lower Perplexity Wins
Generative engines systematically prefer content with lower perplexity, which means higher predictability for the underlying language model. The quantitative effect is substantial: a one standard deviation decrease in perplexity corresponds to an increase in citation probability from 47% to 56%.
This preference emerges from how large language models generate text. At each step, the model predicts the next token based on conditional probability. Content that aligns with these prediction patterns becomes easier for the model to process and cite.
Semantic Homogeneity Among Sources
The second finding is that cited sources demonstrate significantly greater semantic similarity compared to traditionally ranked results. Cited content pairs exhibit 0.0365 higher similarity scores, meaning AI summaries prioritize semantically coherent sources.
What makes this notable is that these preferences do not affect traditional search rankings at all. The researchers found no significant correlation between perplexity and position in conventional search results (coefficient: 0.0144, non-significant). This confirms that generative and traditional search operate through fundamentally different mechanisms.
The Content Polishing Paradox
This is where the research becomes especially relevant for marketers.
The researchers tested what happens when websites undergo LLM-based refinement. They created two treatment conditions: general polishing for clarity and engagement, and goal-oriented polishing explicitly aimed at securing citations.
The surprising result was that AI summaries became more diverse in content sources rather than converging toward homogeneity.
Semantic similarity among cited sources decreased under polishing (β = -0.0319 to -0.0722), while the number of citations expanded by approximately one to two additional sources per query. This contradicts the common concern that AI-driven optimization causes information homogenization.
The mechanism appears to be this: when websites optimize their content to be more predictable and aligned with language model patterns, generative engines can process a wider variety of sources. The algorithm does not narrow its focus; it expands its reach.
Testing Your Content Before Publishing
The research offers a practical approach for marketers who want to optimize for generative engine visibility.
Because citation preferences emerge from intrinsic language model characteristics rather than platform-specific engineering, you can test content using retrieval-augmented generation (RAG) systems built on similar model families.
The researchers demonstrated this by compiling website chunks into PDFs, querying Google’s Gemini RAG API, and measuring citation patterns. The RAG system exhibited identical preferences to the production AI Overview, confirming that publicly available models can predict citation behavior.
This means teams can run offline optimization tests using open or semi-open models before deploying content live. The approach provides a cost-effective way to validate content changes without waiting for search engines to recrawl and reindex pages.
What This Means for Your Brand
The research validates several principles that align with what we have observed at Gumshoe while analyzing thousands of AI-generated conversations.
Content Structure Matters
The perplexity findings suggest that how information is structured directly affects whether AI models cite it. Writing that follows clear, predictable patterns makes it easier for language models to extract and reference expertise.
This does not mean simplifying ideas or removing nuance. It means organizing information in ways that align with how language models process and generate text.
Position Bias Exists
The researchers note that documents exhibit position bias, with earlier content receiving preferential citation consideration. This mirrors traditional SEO wisdom, but the mechanism is different. In generative search, position affects how the model prioritizes information during the retrieval phase rather than how a ranking algorithm orders results.
Semantic Consistency Helps
The semantic homogeneity finding suggests that maintaining consistent terminology and conceptual frameworks across content increases citability. When pages reinforce similar concepts using aligned language, generative engines can more confidently treat a brand as a coherent source.
How Gumshoe Helps You Navigate This Landscape
This research paper explains what generative engines prefer. Gumshoe shows you where your brand appears in AI-generated results and helps you measure the impact of optimization efforts.
Our platform generates thousands of conversations with leading AI models, including ChatGPT, Claude, Perplexity, Google’s AI Overview, and Gemini. This allows teams to see how often their brand is cited, which sources AI models rely on when discussing a category, and how visibility compares to competitors.
The finding that RAG systems mirror production behavior directly validates this approach. By systematically querying AI models and analyzing citation patterns, teams can optimize with evidence rather than guesswork.
When you understand which topics generate citations and which sources AI models prioritize, you can focus effort where it will have the greatest return. Gumshoe’s Brand Visibility Score tracks changes over time so teams can measure whether content improvements actually increase citations.
The Dual Opportunity
One of the most important takeaways from the research is that optimizing for generative engines produces a dual benefit.
Brands can improve citation rates while simultaneously expanding the diversity of information available to users. This is not a zero-sum game. Optimization contributes to a richer information ecosystem rather than degrading it.
The researchers found that higher-educated users completed tasks nearly three minutes faster when encountering polished content, while lower-educated users produced outputs with significantly higher information density. These outcomes suggest that generative engine optimization benefits users as much as brands.
What Comes Next
The researchers note that search platforms may eventually engineer clearer distinctions between RAG APIs and production systems to mitigate manipulation risks. This indicates that the landscape will continue evolving as platforms balance optimization incentives with information quality.
For marketers, the implication is clear: generative engine optimization is a distinct discipline. The mechanisms that determine citations differ from traditional ranking factors, and the techniques that work differ from conventional SEO.
At Gumshoe, we are building the tooling required to operate in this new environment. Our AI Optimization (AIO) recommendations identify concrete improvements that can increase visibility in AI-generated results. Our content generation features help teams create optimized content where their brand currently lacks representation.
The research confirms a core belief: generative search optimization is measurable, testable, and actionable. The algorithms may be complex, but the strategic path forward is increasingly well defined.
By Stan Chang
Head of Product, Gumshoe AI
Sources
When Content Is Goliath and Algorithm Is David: The Style and Semantic Effects of Generative Search Engine
