Transformer-based language models for SMILES strings suffer from a locality gap: standard character-level tokenization fragments chemically meaningful motifs, forcing models to repeatedly learn local syntax at the expense of long-range dependencies. To address this without disrupting standard tokenizers, we propose MolGram, which integrates a conditional $n$-gram memory module into molecular language models. MolGram maps local string patterns to learned embeddings via scalable hash lookups and dynamically injects this regional context into hidden states. Evaluations across three tasks, including unconditional molecule generation, forward reaction prediction, and single-step retrosynthesis, show that MolGram consistently improves performance. Crucially, our analyses demonstrate that MolGram outperforms baselines with 3$\times$ more parameters, establishing explicit local pattern memory as a highly efficient inductive bias.

翻译：基于Transformer的SMILES字符串语言模型面临局部性鸿沟：标准字符级分词破坏了具有化学意义的基序，迫使模型在牺牲长程依赖的情况下反复学习局部语法。为在不干扰标准分词器的情况下解决该问题，我们提出MolGram——一种将条件式$n$-gram记忆模块集成到分子语言模型中的方法。MolGram通过可扩展的哈希查找将局部字符串模式映射为学习嵌入，并将这种区域上下文动态注入隐藏状态。在无条件分子生成、正向反应预测和单步逆合成三项任务上的评估表明，MolGram始终能提升性能。关键分析显示，MolGram能以3倍参数量的优势超越基线模型，确立了显式局部模式记忆作为一种高效归纳偏置的有效性。