What structural inductive bias helps transformers reason over knowledge graphs? Through controlled ablations of a minimal transformer modification with four independently removable components (sparse adjacency masking, edge-type biases, query scaling, value gating), we isolate which structural signals drive multi-hop reasoning. Our finding is sharp: sparse adjacency masking alone accounts for the dominant share of improvement over unmasked transformers (+72.5pp on 3-hop MetaQA, +45.5pp on WebQSP, +53.9pp on CWQ), while learned relation parameters add only modest refinement and can actively hurt without structural guidance. A zero-shot experiment provides architecturally independent corroboration: masking-based attention degrades 4.0x less than relation-specific weights when edge types are held out. The useful inductive bias for multi-hop KGQA is predominantly topological, not relational.

翻译：什么结构归纳偏置能够帮助Transformer在知识图谱上进行推理？通过对一个最小化Transformer修改（包含四个独立可移除组件：稀疏邻接掩码、边类型偏置、查询缩放、值选通）进行受控消融实验，我们分离出驱动多跳推理的结构信号。研究结果明确：稀疏邻接掩码单独贡献了相较于未掩码Transformer的主要性能提升（3跳MetaQA提升72.5个百分点，WebQSP提升45.5个百分点，CWQ提升53.9个百分点），而学习到的关系参数仅带来适度改进，且在缺乏结构引导时会产生负面影响。一项零样本实验提供了架构独立的佐证：当边类型被剔除时，基于掩码的注意力机制退化程度比关系特定权重低4.0倍。多跳KGQA的有用归纳偏置主要是拓扑性的，而非关系性的。