DOM-Aware Chunking: How OpenClaw Parses HTML Structure

When a human looks at a webpage, they don’t see code. They see a headline, a sidebar, a main article, and a footer. They intuitively group related information together based on visual cues: whitespace, font size, border lines, and background colors.

When a standard RAG pipeline looks at a webpage, it sees a flat string of text. It sees <h1> and <p> tags mashed together, stripped of their spatial context. It sees the “Related Articles” sidebar as just another paragraph in the middle of the main content.

In the early days of the web, we were told to use Semantic HTML for accessibility. We were told it allowed screen readers to navigate our content, providing a better experience for the visually impaired. We were told it might help SEO, though Google’s engineers were always famously coy about whether an <article> tag carried significantly more weight than a well-placed <div>.

In 2025, that game has changed entirely. We are no longer just optimizing for screen readers or the ten blue links on a search results page. We are optimizing for the training sets of Large Language Models (LLMs).