Evaluating block-diagonal curvature models in neural networks involves two questions: how much curvature they omit, and what a specific correction changes at a finite step. We measure the squared off-block fraction of the parameter Hessian against references for group sizes and the observed within-group curvature profile. In CIFAR-100 ResNet-18 and a skip-free counterpart, similar initial discarded fractions accompany different profiles. At saved trained ResNet-18 weights, changing from stage to layer groups moves estimated off-block mass from below to above the profile reference. An approximate assessment of probe uncertainty conditional on fixed weights and inputs resolves this crossing; some skip-free comparisons remain unresolved. Separately, in nine saved ResNet-50/Stanford Cars states from three training seeds, we compare candidates formed from off-group Hessian action, the remaining model residual and their combination. For each state, we construct two full-Hessian-informed maps on separate geometry draws. Within each draw, all three inputs share one map. At unit scale off-group input alone raises evaluated minibatch loss relative to the base preconditioner candidate in 17 of 18 state/draw comparisons, yet the combined candidate has lower loss than the remainder-only candidate in nine of those cases; preferences vary with scale and correction-norm matching. The profiles help assess block models and alternative groupings, while same-state candidate comparisons test a particular correction without equating structural mass with training gain.
翻译:暂无翻译