Language models are widely used in chemistry for molecular property prediction and small-molecule generation, yet Natural Products (NPs) remain underexplored despite their importance in drug discovery. To address this gap, we develop NP-specific chemical language models (NPCLMs) by pre-training state-space models (Mamba and Mamba-2) and comparing them with transformer baselines (GPT). Using a dataset of about 1M NPs, we present the first systematic comparison of selective state-space models and transformers for NP-focused tasks, together with eight tokenization strategies including character-level, Atom-in-SMILES (AIS), byte-pair encoding (BPE), and NP-specific BPE. We evaluate molecule generation (validity, uniqueness, novelty) and property prediction (membrane permeability, taste, anti-cancer activity) using MCC and AUC-ROC. Mamba generates 1-2 percent more valid and unique molecules than Mamba-2 and GPT, with fewer long-range dependency errors, while GPT yields slightly more novel structures. For property prediction, Mamba variants outperform GPT by 0.02-0.04 MCC under random splits, while scaffold splits show comparable performance. Results demonstrate that domain-specific pre-training on about 1M NPs can match models trained on datasets over 100 times larger.

翻译：语言模型在化学领域广泛应用于分子性质预测与小分子生成，然而天然产物在药物发现中的重要性虽高，其研究仍显不足。为填补这一空白，我们通过预训练状态空间模型（Mamba与Mamba-2）并与Transformer基线模型（GPT）进行比较，开发了针对天然产物的化学语言模型。基于约100万天然产物的数据集，本研究首次系统比较了选择性状态空间模型与Transformer在天然产物相关任务上的表现，同时评估了包括字符级、SMILES原子编码、字节对编码及天然产物专用字节对编码在内的八种分词策略。我们采用马修斯相关系数与受试者工作特征曲线下面积评估了分子生成（有效性、独特性、新颖性）与性质预测（膜渗透性、味觉特征、抗癌活性）。Mamba模型生成的有效且独特分子比Mamba-2与GPT多1-2%，且长程依赖错误更少，而GPT生成的结构略具更高新颖性。在性质预测方面，随机分割下Mamba变体比GPT的MCC值高0.02-0.04，骨架分割下两者性能相当。结果表明，针对约100万天然产物的领域特异性预训练可达到在百倍规模数据集上训练模型的同等效果。