This post is the second half of a two-part argument. The first half examined llms.txt advertising and found it fails on delivery — the file almost never gets fetched, so the impression never happens. This half examines an advertising product where delivery is solved. The failure is one layer up. Disclosure: this site builds and operates agent-detection infrastructure. The verification gap described in the close is one this operation would benefit from commercially. The analysis is published because the argument follows from the evidence, not from the incentive.

An AI Overview once reported that the cat on tamethebots.com held a job title. The cat’s name was Odd. The role was invented. The citation pointed to a plain-text file with no schema, no author field, and no inbound links — a file that had been live for roughly two weeks. That detail circulated widely. Less widely circulated: whether anyone else could reproduce it, where it came from, and what, precisely, it proves.

Imagine a rate card that does not yet exist. For $250 per month, your product appears in the recommended-tools section of a documentation site’s llms.txt, which is parsed by coding agents at the moment a developer asks what library to use. The agent fetches the file, reads the description string, and your product lands in context at the exact instant the decision is made. This is a coherent-sounding product. Adjacent versions are already being sold: paid brand mentions in AI Overviews, sponsored inclusions in AI-generated listicles, and placement in curated “agent context” resources. DiNardi’s audit in Search Engine Land found placements at $250 per mention, with PBN-style inventory selling at 10–15x backlink rates. The rate card for llms.txt is merely the logical next step.

In the hierarchy of web crawlers, there is Googlebot, there is Bingbot, and then there is OpenClaw. While traditional search engine bots are polite librarians cataloging books, OpenClaw is a voracious scholar tearing pages out to build a new compendium. OpenClaw is an Autonomous Research Agent. It doesn’t just index URLs; it traverses the web to synthesize knowledge graphs. If your site blocks OpenClaw, you aren’t just missing from a search engine results page; you are missing from the collective intelligence of the Agentic Web.

Google Search Console (GSC) has historically been the dashboard of record for SEOs. But in the agentic era, GSC is becoming a lagging indicator. It often fails to report on the activity of new AI agents, RAG bots, and specialized crawlers. To truly understand how the AI ecosystem views your site, you must return to the source: Server Logs. The Limitations of GSC GSC is designed for Google Search. It tells you little about how ChatGPT (OpenAI), Claude (Anthropic), or Perplexity are interacting with your site. If GPTBot fails to crawl your site due to a firewall rule, GSC will never tell you.

“PageRank” is the zombie concept of SEO. It refuses to die, shambling through every forum thread and conference slide deck for 25 years. But in 2025, when checking your “Crawled - currently not indexed” report, invoking PageRank is worse than useless—it is misleading. The classical definition of PageRank was a probability distribution: the likelihood that a random surfer would land on a page. Today, the metric that matters is Indexing Probability.

There is a comforting lie that SEOs tell themselves when they see the dreaded “Crawled - currently not indexed” status in Google Search Console (GSC). The lie is: “My content just needs to be better.” We audit the page. We add more H2s. We add a video. We “optimize” the meta description. And then we wait. And it stays not indexed. The uncomfortable truth of 2025 is that indexing is no longer a meritocracy of quality; it is a calculation of marginal utility. Google is not rejecting your page because it is “bad.” Google is rejecting your page because indexing it costs more in electricity and storage than it will ever generate in ad revenue.