A recent article by Dejan.ai titled “Google Just Quietly Dropped the Biggest Shift in Technical SEO” has been making the rounds. While we respect Dejan’s history in the industry, their analysis of WebMCP suffers from a classic “Web 2.0” bias.
They view WebMCP primarily as a Discovery Mechanism. We argue it is an Execution Mechanism. And that distinction changes everything.
What is WebMCP?
For the uninitiated, vast confusion surrounds this term.
Read more →As we build the Agentic Web, a confusing alphabet soup of standards is emerging. Three files, in particular, are vying for the attention of modern SEOs: llms.txt, cats.txt, and the new WebMCP protocol.
They often get confused, but they serve three distinct purposes in the lifecycle of an AI interaction. Think of them as Context, Contract, and Capability.
1. LLMS.TXT: The Context (What to Know)
- Role: Documentation for Robots.
- Location: Root directory (
/llms.txt). - Audience: Training crawlers and RAG agents.
llms.txt is essentially a Markdown file that strips away the HTML “cruft” of your website. It provides a clean, token-efficient summary of your content. It answers the question: “What information does this website hold?”
Read more →The ethical debate around AI training data is fierce. “They stole our content!” is the cry of publishers. “It was fair use!” is the retort of AI labs. CATS (Content Authorization & Transparency Standard) is the technical solution to this legal standoff.
Implementing CATS is not just about blocking bots; it is about establishing a contract.
The CATS Workflow
- Discovery: The agent checks
/.well-known/cats.json or cats.txt at the root. - Negotiation: The agent parses your policy.
- “Can I index this?” -> Yes.
- “Can I train on this?” -> No.
- “Can I display a snippet?” -> Yes, max 200 chars.
- “Do I need to pay?” -> Check
pricing object.
- Compliance: The agent (if ethical) respects these boundaries.
Signaling “Cooperative Node” Status
Search engines of the future constitutes a “Web of Trust.” Sites that implement CATS are signaling that they are “Cooperative Nodes.” They are providing clear metadata about their rights.
Read more →For nearly three decades, the robots.txt file has served as the internet’s “Keep Out” sign. It is a binary, blunt instrument: Allow or Disallow. Crawlers either respect it or they don’t. However, as we enter the age of the Agentic Web, this binary distinction is no longer sufficient. We need a protocol that can express nuance, permissions, licenses, and economic terms. We need CATS (Content Authorization & Transparency Standard), often implemented as cats.txt or authorized_agents.json.
Read more →