A Critical Review of Dejan.ai's WebMCP Analysis

A recent article by Dejan.ai titled “Google Just Quietly Dropped the Biggest Shift in Technical SEO” has been making the rounds. While we respect Dejan’s history in the industry, their analysis of WebMCP suffers from a classic “Web 2.0” bias.

They view WebMCP primarily as a Discovery Mechanism. We argue it is an Execution Mechanism. And that distinction changes everything.

What is WebMCP?

For the uninitiated, vast confusion surrounds this term.

Read more →

The Agentic Trilogy: LLMS.TXT, CATS.TXT, and WebMCP

As we build the Agentic Web, a confusing alphabet soup of standards is emerging. Three files, in particular, are vying for the attention of modern SEOs: llms.txt, cats.txt, and the new WebMCP protocol.

They often get confused, but they serve three distinct purposes in the lifecycle of an AI interaction. Think of them as Context, Contract, and Capability.

1. LLMS.TXT: The Context (What to Know)

  • Role: Documentation for Robots.
  • Location: Root directory (/llms.txt).
  • Audience: Training crawlers and RAG agents.

llms.txt is essentially a Markdown file that strips away the HTML “cruft” of your website. It provides a clean, token-efficient summary of your content. It answers the question: “What information does this website hold?”

Read more →

Implementing CATS Protocols for Ethical Scraping

The ethical debate around AI training data is fierce. “They stole our content!” is the cry of publishers. “It was fair use!” is the retort of AI labs. CATS (Content Authorization & Transparency Standard) is the technical solution to this legal standoff.

Implementing CATS is not just about blocking bots; it is about establishing a contract.

The CATS Workflow

  1. Discovery: The agent checks /.well-known/cats.json or cats.txt at the root.
  2. Negotiation: The agent parses your policy.
    • “Can I index this?” -> Yes.
    • “Can I train on this?” -> No.
    • “Can I display a snippet?” -> Yes, max 200 chars.
    • “Do I need to pay?” -> Check pricing object.
  3. Compliance: The agent (if ethical) respects these boundaries.

Signaling “Cooperative Node” Status

Search engines of the future constitutes a “Web of Trust.” Sites that implement CATS are signaling that they are “Cooperative Nodes.” They are providing clear metadata about their rights.

Read more →

CATS.TXT: The Constitution for Autonomous Agents

For nearly three decades, the robots.txt file has served as the internet’s “Keep Out” sign. It is a binary, blunt instrument: Allow or Disallow. Crawlers either respect it or they don’t. However, as we enter the age of the Agentic Web, this binary distinction is no longer sufficient. We need a protocol that can express nuance, permissions, licenses, and economic terms. We need CATS (Content Authorization & Transparency Standard), often implemented as cats.txt or authorized_agents.json.

Read more →