Current abstractive summarization models either suffer from a lack of clear interpretability or provide incomplete rationales by only highlighting parts of the source document. To this end, we propose the Summarization Program (SP), an interpretable modular framework consisting of an (ordered) list of binary trees, each encoding the step-by-step generative process of an abstractive summary sentence from the source document. A Summarization Program contains one root node per summary sentence, and a distinct tree connects each summary sentence (root node) to the document sentences (leaf nodes) from which it is derived, with the connecting nodes containing intermediate generated sentences. Edges represent different modular operations involved in summarization such as sentence fusion, compression, and paraphrasing. We first propose an efficient best-first search method over neural modules, SP-Search that identifies SPs for human summaries by directly optimizing for ROUGE scores. Next, using these programs as automatic supervision, we propose seq2seq models that generate Summarization Programs, which are then executed to obtain final summaries. We demonstrate that SP-Search effectively represents the generative process behind human summaries using modules that are typically faithful to their intended behavior. We also conduct a simulation study to show that Summarization Programs improve the interpretability of summarization models by allowing humans to better simulate model reasoning. Summarization Programs constitute a promising step toward interpretable and modular abstractive summarization, a complex task previously addressed primarily through blackbox end-to-end neural systems. Supporting code available at https://github.com/swarnaHub/SummarizationPrograms

翻译：当前抽象摘要模型要么缺乏清晰的解释性，要么仅通过突出源文档部分内容提供不完整的理由。为此，我们提出摘要程序（SP），这是一个可解释的模块化框架，由（有序的）二叉树列表组成，每个树编码从源文档生成抽象摘要句子的逐步生成过程。摘要程序为每个摘要句子包含一个根节点，并通过不同的树将每个摘要句子（根节点）与其来源的文档句子（叶节点）连接起来，连接节点包含中间生成的句子。边表示摘要过程中涉及的不同模块化操作，如句子融合、压缩和改写。我们首先提出一种高效的基于神经模块的最佳优先搜索方法——SP-Search，通过直接优化ROUGE分数来识别人工摘要的SP。接下来，利用这些程序作为自动监督，我们提出生成摘要程序的序列到序列模型，然后执行这些模型以获得最终摘要。我们证明SP-Search能够有效使用通常忠实于其预期行为的模块来表示人工摘要背后的生成过程。我们还进行了一项模拟研究，表明摘要程序通过允许人类更好地模拟模型推理来提高摘要模型的可解释性。摘要程序为可解释和模块化的抽象摘要（这一此前主要通过黑盒端到端神经系统的复杂任务）迈出了有希望的一步。支持代码见：https://github.com/swarnaHub/SummarizationPrograms