Chaining language model (LM) calls as composable modules is fueling a new powerful way of programming. However, ensuring that LMs adhere to important constraints remains a key challenge, one often addressed with heuristic "prompt engineering". We introduce LM Assertions, a new programming construct for expressing computational constraints that LMs should satisfy. We integrate our constructs into the recent DSPy programming model for LMs, and present new strategies that allow DSPy to compile programs with arbitrary LM Assertions into systems that are more reliable and more accurate. In DSPy, LM Assertions can be integrated at compile time, via automatic prompt optimization, and/or at inference time, via automatic selfrefinement and backtracking. We report on two early case studies for complex question answering (QA), in which the LM program must iteratively retrieve information in multiple hops and synthesize a long-form answer with citations. We find that LM Assertions improve not only compliance with imposed rules and guidelines but also enhance downstream task performance, delivering intrinsic and extrinsic gains up to 35.7% and 13.3%, respectively. Our reference implementation of LM Assertions is integrated into DSPy at https://github.com/stanfordnlp/dspy

翻译：将语言模型（LM）调用串联为可组合模块，正催生一种强大的新型编程方式。然而，确保LM遵循重要约束仍是一项关键挑战，通常需借助启发式"提示工程"加以应对。我们提出LM断言这一新型编程结构，用于表达LM应满足的计算约束。我们将该结构融入近期提出的DSPy LM编程模型，并呈现新策略使DSPy能将包含任意LM断言的程序编译为更可靠、更精确的系统。在DSPy中，LM断言可通过自动提示优化在编译时集成，或通过自动自精炼与回溯在推理时集成。我们针对复杂问答（QA）任务报告两项早期案例研究，其中LM程序需迭代多跳检索信息，并综合形成附引文的详细答案。研究发现LM断言不仅提升对既定规则与指南的遵循度，更增强下游任务性能，带来分别高达35.7%和13.3%的内在增益与外在增益。我们的LM断言参考实现已集成至DSPy，代码仓库为https://github.com/stanfordnlp/dspy。