Many types of data from fields including natural language processing, computer vision, and bioinformatics, are well represented by discrete, compositional structures such as trees, sequences, or matchings. Latent structure models are a powerful tool for learning to extract such representations, offering a way to incorporate structural bias, discover insight about the data, and interpret decisions. However, effective training is challenging, as neural networks are typically designed for continuous computation. This text explores three broad strategies for learning with discrete latent structure: continuous relaxation, surrogate gradients, and probabilistic estimation. Our presentation relies on consistent notations for a wide range of models. As such, we reveal many new connections between latent structure learning strategies, showing how most consist of the same small set of fundamental building blocks, but use them differently, leading to substantially different applicability and properties.

翻译：在自然语言处理、计算机视觉和生物信息学等诸多领域中，许多类型的数据都能很好地用离散的组合结构（如树、序列或匹配）来表示。潜在结构模型是学习提取此类表示的有力工具，它提供了一种纳入结构偏置、发现数据洞见并解释决策的途径。然而，由于神经网络通常是为连续计算而设计的，因此有效训练具有挑战性。本文探讨了学习离散潜在结构的三大策略：连续松弛、替代梯度和概率估计。我们的阐述采用了一套适用于多种模型的一致符号体系。由此，我们揭示了潜在结构学习策略之间的诸多新联系，表明大多数策略都由相同的小型基本构建模块组成，但使用方式不同，从而导致其适用性和性质存在显著差异。