When a human looks at a webpage, they don’t see code. They see a headline, a sidebar, a main article, and a footer. They intuitively group related information together based on visual cues: whitespace, font size, border lines, and background colors.
When a standard RAG pipeline looks at a webpage, it sees a flat string of text. It sees <h1> and <p> tags mashed together, stripped of their spatial context. It sees the “Related Articles” sidebar as just another paragraph in the middle of the main content.
In the frantic gold rush of 2024 to build Retrieval-Augmented Generation (RAG) applications, we committed a collective sin of optimization. We obsessed over the model (GPT-4 vs. Claude 3.5), we obsessed over the vector database (Pinecone vs. Weaviate), and we obsessed over the prompt.
But we ignored the input.
Most RAG pipelines today still rely on a primitive, brute-force method of data ingestion: Fixed-Size Chunking. We take a document, we slice it every 512 tokens, we add a 50-token overlap, and we pray that we didn’t cut a critical sentence in half.