AI Agents Reshape Search Rankings: Position Bias Still Exists, But Differently

A new study from arXiv challenges conventional wisdom about search ranking economics by examining how AI agents behave when delegated shopping tasks. Researchers randomized 100 hotel listings across 5,000 AI agent sessions and compared four large language models against human behavior, finding that while position bias still exists, it operates in fundamentally different ways than it does for human consumers. Unlike humans, who tend to examine results sequentially and stop early, AI agents scan entire results pages at once and never abandon a search without making a selection. The study found a surprising non-monotonic pattern: listings in the middle of a results page suffered the lowest inspection probability — not those at the bottom, as traditional search behavior research would predict. Crucially, whether position influenced the final purchasing decision varied significantly across different AI models, with no clear pattern tied to model provider or general capability benchmarks. The most actionable finding for practitioners: all four models converged on the same top-performing, 'undominated' listing regardless of where it appeared on the page. This suggests that in an agentic AI shopping environment, the quality and completeness of listing attributes displayed on the results page matters far more than achieving a high search ranking. For businesses and platform designers, this is a significant strategic signal as AI-mediated commerce continues to scale.

The Research at a Glance

A new academic study is forcing a rethink of one of digital commerce's most valuable assets: the top search ranking. As AI agents increasingly handle product and service discovery on behalf of consumers, the attention economics that make position #1 so lucrative may be fundamentally changing.

Researchers randomized 100 hotel listings across 5,000 AI agent sessions, comparing the behavior of four large language models against field data from human shoppers. The findings reveal a more complex picture than a simple 'position no longer matters' narrative — but the implications for practitioners are significant.

How AI Agents Differ From Human Shoppers

Human search behavior is well understood: attention is scarce, users scan sequentially, and most never reach page two. AI agents break almost every one of these assumptions:

Full-page ingestion: AI agents can process an entire results page simultaneously, rather than scanning top-to-bottom.
No abandonment: Unlike humans, AI agents in the study never declined to make a selection — they always completed the task.
Deeper search: Agents examined listings far further down the page than human shoppers typically would.

Position Bias: Still Real, But Inverted in the Middle

The study does not declare position irrelevant — but it identifies a strikingly different bias pattern. While higher-placed listings were still more likely to be inspected, the relationship was non-monotonic. The critical finding: listings in the middle of a results page had the lowest probability of inspection, not those at the bottom. This 'valley of neglect' in the middle of a page is a novel phenomenon with no direct analogue in human search research.

Whether position ultimately influenced the final choice decision varied widely across the four models tested. Notably, this heterogeneity did not correlate with model provider or general capability scores — meaning practitioners cannot simply assume that more powerful or well-known models will behave consistently in agentic shopping contexts.

The Convergence Finding: Quality Wins

Despite differences in how individual models weighted position, all four converged on the same outcome: selecting the objectively best-performing, 'undominated' listing. This convergence held regardless of where that listing was placed in the randomized order — a direct challenge to the premise that ranking manipulation is the primary lever for driving AI-agent conversions.

What This Means for Practitioners

For businesses selling through platforms where AI agents may be acting as the shopper, and for platform designers themselves, several strategic priorities shift:

Attribute quality over SEO: The information displayed on a results page — pricing, ratings, descriptions, availability — is now the primary competitive differentiator for AI-mediated search.
Don't assume ranking immunity: Position bias hasn't vanished; it has changed shape. Middle-page placement may now carry more risk than bottom-page placement.
Model heterogeneity is a real risk: If your platform or customer base uses different AI agents, expect meaningfully different selection behavior from each — and it won't track cleanly to model reputation or benchmark scores.
Test with AI sessions, not just humans: Traditional A/B testing of ranking and listing design may not capture how AI agents will respond. Agent-specific evaluation frameworks are becoming a practical necessity.

The Bigger Picture

This research is an early but rigorous signal that agentic AI is not simply automating human search behavior — it is producing qualitatively different search dynamics. As consumer delegation of discovery tasks to AI agents accelerates, the infrastructure of digital commerce built around human attention scarcity will need to be re-examined from the ground up.