This paper reviews the state-of-the-art of language models architectures and strategies for "complex" question-answering (QA, CQA, CPS) with a focus on hybridization. Large Language Models (LLM) are good at leveraging public data on standard problems but once you want to tackle more specific complex questions or problems (e.g. How does the concept of personal freedom vary between different cultures ? What is the best mix of power generation methods to reduce climate change ?) you may need specific architecture, knowledge, skills, methods, sensitive data protection, explainability, human approval and versatile feedback... Recent projects like ChatGPT and GALACTICA have allowed non-specialists to grasp the great potential as well as the equally strong limitations of LLM in complex QA. In this paper, we start by reviewing required skills and evaluation techniques. We integrate findings from the robust community edited research papers BIG, BLOOM and HELM which open source, benchmark and analyze limits and challenges of LLM in terms of tasks complexity and strict evaluation on accuracy (e.g. fairness, robustness, toxicity, ...) as a baseline. We discuss some challenges associated with complex QA, including domain adaptation, decomposition and efficient multi-step QA, long form and non-factoid QA, safety and multi-sensitivity data protection, multimodal search, hallucinations, explainability and truthfulness, temporal reasoning. We analyze current solutions and promising research trends, using elements such as: hybrid LLM architectural patterns, training and prompting strategies, active human reinforcement learning supervised with AI, neuro-symbolic and structured knowledge grounding, program synthesis, iterated decomposition and others.

翻译：本文综述了面向“复杂”问答（QA、CQA、CPS）的语言模型架构与策略，重点关注混合化方法。大型语言模型（LLM）擅长利用标准问题的公开数据，但当应对更具体的复杂问题（例如：不同文化间个人自由概念如何变化？何种发电方式组合最能缓解气候变化？）时，可能需要特定架构、知识、技能、方法、敏感数据保护、可解释性、人工审核及多维度反馈。近期ChatGPT和GALACTICA等项目使非专业人士认识到LLM在复杂问答中的巨大潜力与同样显著的局限性。本文首先梳理所需技能与评估技术，整合社区编排的研究论文BIG、BLOOM和HELM的成果，这些开源基准分析了LLM在任务复杂度和严格准确性评估（如公平性、鲁棒性、毒性等）方面的限制与挑战。我们讨论了复杂问答中的若干挑战，包括领域适应、分解与高效多步问答、长文本与非事实型问答、安全与多敏感数据保护、多模态检索、幻觉问题、可解释性与真实性、时序推理。通过分析混合LLM架构模式、训练与提示策略、AI监督的主动人类强化学习、神经符号与结构化知识锚定、程序合成、迭代分解等要素，我们探讨了当前解决方案与具有前景的研究趋势。