How do protein structure prediction models fold proteins? We investigate this question by tracing how ESMFold folds a beta hairpin, a prevalent structural motif. Through counterfactual interventions on model latents, we identify two computational stages in the folding trunk. In the first stage, early blocks initialize pairwise biochemical signals: residue identities and associated biochemical features such as charge flow from sequence representations into pairwise representations. In the second stage, late blocks develop pairwise spatial features: distance and contact information accumulate in the pairwise representation. We demonstrate that the mechanisms underlying structural decisions of ESMFold can be localized, traced through interpretable representations, and manipulated with strong causal effects.

翻译：蛋白质结构预测模型如何折叠蛋白质？我们通过追踪ESMFold折叠β发夹（一种普遍存在的结构基序）的过程来研究这一问题。通过对模型潜在空间进行反事实干预，我们在折叠主干中识别出两个计算阶段。第一阶段中，早期模块初始化成对生化信号：残基身份及其相关生化特征（如电荷）从序列表征流向成对表征。第二阶段中，晚期模块发展成对空间特征：距离与接触信息在成对表征中逐步积累。我们证明ESMFold结构决策的机制可以被定位、通过可解释表征进行追踪，并能通过强因果效应进行操控。