Protein language models (pLMs) have emerged as powerful predictors of protein structure and function. However, the computational circuits underlying their predictions remain poorly understood. Recent mechanistic interpretability methods decompose pLM representations into interpretable features, but they treat each layer independently and thus fail to capture cross-layer computation, limiting their ability to approximate the full model. We introduce ProtoMech, a framework for discovering computational circuits in pLMs using cross-layer transcoders that learn sparse latent representations jointly across layers to capture the model's full computational circuitry. Applied to the pLM ESM2, ProtoMech recovers 82-89% of the original performance on protein family classification and function prediction tasks. ProtoMech then identifies compressed circuits that use <1% of the latent space while retaining up to 79% of model accuracy, revealing correspondence with structural and functional motifs, including binding, signaling, and stability. Steering along these circuits enables high-fitness protein design, surpassing baseline methods in more than 70% of cases. These results establish ProtoMech as a principled framework for protein circuit tracing.

翻译：蛋白质语言模型（pLM）已成为预测蛋白质结构与功能的有力工具。然而，其预测背后的计算回路机制仍不甚明晰。现有的机制可解释性方法虽能将pLM表征分解为可解释特征，但因其独立处理各网络层而无法捕捉跨层计算过程，从而限制了完整模型的近似能力。本文提出ProtoMech框架，通过跨层转码器学习稀疏潜表征的联合跨层分布，以此捕捉模型完整计算回路，实现pLM计算回路的系统性发现。在ESM2蛋白质语言模型上的应用表明，ProtoMech在蛋白质家族分类与功能预测任务中恢复了原模型82-89%的性能。该框架进一步识别出仅占用潜空间<1%的压缩计算回路，同时保持高达79%的模型精度，这些回路与结合、信号传导及稳定性等结构功能基序存在对应关系。沿这些回路进行定向调控可实现高适应性蛋白质设计，在超过70%的案例中超越基准方法。研究结果确立了ProtoMech作为蛋白质回路追踪的规范化框架。