Retrieving answers in a quick and low cost manner without hallucinations from a combination of structured and unstructured data using Language models is a major hurdle which prevents employment of Language models in knowledge retrieval automation. This becomes accentuated when one wants to integrate a speech interface. Besides, for commercial search and chatbot applications, complete reliance on commercial large language models (LLMs) like GPT 3.5 etc. can be very costly. In this work, authors have addressed this problem by first developing a keyword based search framework which augments discovery of the context to be provided to the large language model. The keywords in turn are generated by LLM and cached for comparison with keywords generated by LLM against the query raised. This significantly reduces time and cost to find the context within documents. Once the context is set, LLM uses that to provide answers based on a prompt tailored for Q&A. This research work demonstrates that use of keywords in context identification reduces the overall inference time and cost of information retrieval. Given this reduction in inference time and cost with the keyword augmented retrieval framework, a speech based interface for user input and response readout was integrated. This allowed a seamless interaction with the language model.

翻译：从结构化和非结构化数据的组合中，以快速、低成本的方式且无幻觉地利用语言模型检索答案，是阻碍语言模型在知识检索自动化中应用的主要障碍。当需要集成语音接口时，这一挑战更加突出。此外，对于商业搜索和聊天机器人应用，完全依赖如GPT 3.5等商用大语言模型（LLMs）可能代价高昂。在本工作中，作者通过首先开发一个基于关键词的检索框架来解决该问题，该框架增强了对需提供给大语言模型的上下文的发现。关键词由大语言模型生成并缓存，用于与针对查询生成的关键词进行对比。这显著减少了在文档中定位上下文的时间和成本。一旦上下文确定，大语言模型便基于为问答定制的提示指令，利用该上下文提供答案。本研究表明，在上下文识别中使用关键词可降低信息检索的整体推理时间和成本。鉴于关键词增强检索框架在推理时间和成本上的降低，本文集成了用于用户输入和响应朗读的语音接口，从而实现了与语言模型的无缝交互。