We have spent the last decade complaining about “Crawled - currently not indexed.” We treat it as a failure state. We treat it as a bug.

But in the Agentic Web of 2025, “Indexation” is not the goal. “Retrieval” is the goal.

And paradoxically, to maximize Retrieval, you often need to minimize Indexation.

The Information Density Argument

LLMs (Large Language Models) and Search Agents operate on Information Density. They want the highest signal-to-noise ratio possible.