The quality of data representation is paramount for the performance of a model. Recent research has focused on enhancing representation learning by incorporating more information about the intra-sample structures of individual data points, such as local and global attention. Additionally, researchers have explored methods to model the inter-sample relationships, including manifold, contrastive, and discrete representation learning. In this study, we introduce a new training loss, which considers both intra-sample structure and inter-sample relationships, leveraging the concept of {\it atoms} to represent data points. This new approach, {\it Atom Modeling}, offers a fresh perspective to discretize data representations within a continuous space. Through experiments, we demonstrate that Atom Modeling enhances the performance of existing models in tasks involving classification and generation, across diverse domains including vision and language. These findings underscore the potential of Atom Modeling to enhance data representation and improve model learning, suggesting a promising direction for future research.

翻译：数据表示的质量对模型性能至关重要。近年来，研究聚焦于通过融入更多关于单个数据点内部结构的信息（如局部与全局注意力）来增强表示学习。此外，研究人员还探索了建模样本间关系的方法，包括流形学习、对比学习和离散表示学习。在本研究中，我们引入了一种新的训练损失函数，同时兼顾样本内结构与样本间关系，利用"原子"概念来表示数据点。这种名为"原子建模"的新方法，为在连续空间中离散化数据表示提供了全新视角。实验证明，原子建模能提升现有模型在分类和生成任务中的性能，涵盖视觉和语言等多个领域。这些发现揭示了原子建模在增强数据表示、优化模型学习方面的潜力，为未来研究指明了一个有前景的方向。