Retrieving answers in a quick and low cost manner without hallucinations from a combination of structured and unstructured data using Language models is a major hurdle. This is what prevents employment of Language models in knowledge retrieval automation. This becomes accentuated when one wants to integrate a speech interface on top of a text based knowledge retrieval system. Besides, for commercial search and chat-bot applications, complete reliance on commercial large language models (LLMs) like GPT 3.5 etc. can be very costly. In the present study, the authors have addressed the aforementioned problem by first developing a keyword based search framework which augments discovery of the context from the document to be provided to the LLM. The keywords in turn are generated by a relatively smaller LLM and cached for comparison with keywords generated by the same smaller LLM against the query raised. This significantly reduces time and cost to find the context within documents. Once the context is set, a larger LLM uses that to provide answers based on a prompt tailored for Q\&A. This research work demonstrates that use of keywords in context identification reduces the overall inference time and cost of information retrieval. Given this reduction in inference time and cost with the keyword augmented retrieval framework, a speech based interface for user input and response readout was integrated. This allowed a seamless interaction with the language model.

翻译：从结构化与非结构化数据的组合中，利用语言模型快速、低成本且无幻觉地检索答案，是一项重大挑战。这阻碍了语言模型在知识检索自动化中的应用。当需要在基于文本的知识检索系统之上集成语音接口时，这一问题更为突出。此外，对于商业搜索和聊天机器人应用，完全依赖如GPT 3.5等商业大语言模型（LLM）可能成本极高。在本研究中，作者通过首先开发一个基于关键词的搜索框架来解决上述问题，该框架增强了对文档中上下文的发现，并将上下文提供给LLM。关键词由相对较小的LLM生成并缓存，用于与同一较小LLM针对查询生成的关键词进行比较。这显著降低了在文档中查找上下文的时间和成本。在确定上下文后，较大的LLM利用该上下文，基于针对问答定制的提示提供答案。本研究表明，在上下文识别中使用关键词可降低信息检索的整体推理时间和成本。鉴于关键词增强检索框架在推理时间和成本上的降低，本研究集成了用于用户输入和响应语音读出的语音接口，实现了与语言模型的无缝交互。