Retrieval-Augmented Generation (RAG) has been shown to improve knowledge capabilities and alleviate the hallucination problem of LLMs. The Web is a major source of external knowledge used in RAG systems, and many commercial systems such as ChatGPT and Perplexity have used Web search engines as their major retrieval systems. Typically, such RAG systems retrieve search results, download HTML sources of the results, and then extract plain texts from the HTML sources. Plain text documents or chunks are fed into the LLMs to augment the generation. However, much of the structural and semantic information inherent in HTML, such as headings and table structures, is lost during this plain-text-based RAG process. To alleviate this problem, we propose HtmlRAG, which uses HTML instead of plain text as the format of retrieved knowledge in RAG. We believe HTML is better than plain text in modeling knowledge in external documents, and most LLMs possess robust capacities to understand HTML. However, utilizing HTML presents new challenges. HTML contains additional content such as tags, JavaScript, and CSS specifications, which bring extra input tokens and noise to the RAG system. To address this issue, we propose HTML cleaning, compression, and pruning strategies, to shorten the HTML while minimizing the loss of information. Specifically, we design a two-step block-tree-based pruning method that prunes useless HTML blocks and keeps only the relevant part of the HTML. Experiments on six QA datasets confirm the superiority of using HTML in RAG systems.

翻译：检索增强生成（RAG）已被证明能够提升大语言模型的知识能力并缓解其幻觉问题。网络是RAG系统中外部知识的主要来源，许多商业系统（如ChatGPT和Perplexity）已采用网络搜索引擎作为其主要检索系统。通常，此类RAG系统会检索搜索结果、下载结果的HTML源码，然后从HTML源码中提取纯文本。纯文本文档或文本块被输入大语言模型以增强生成过程。然而，HTML中固有的许多结构和语义信息（如标题和表格结构）在此基于纯文本的RAG过程中丢失。为缓解此问题，我们提出HtmlRAG，它使用HTML而非纯文本作为RAG中检索知识的格式。我们认为HTML在建模外部文档知识方面优于纯文本，且大多数大语言模型具备理解HTML的强大能力。然而，使用HTML也带来了新的挑战。HTML包含标签、JavaScript和CSS规范等额外内容，这些内容会给RAG系统带来额外的输入标记和噪声。为解决这一问题，我们提出了HTML清洗、压缩和剪枝策略，以缩短HTML长度同时最小化信息损失。具体而言，我们设计了一种基于块树的两步剪枝方法，该方法剪除无用的HTML块并仅保留HTML的相关部分。在六个问答数据集上的实验证实了在RAG系统中使用HTML的优越性。