Masked prediction learns to infer missing variables from visible context selected by a mask schedule. When does small excess risk guarantee recovery of the true joint distribution? We study this question on finite product spaces under masked-block log loss, with conditionals induced by a single joint distribution. To quantify recovery, we introduce an $\varepsilon$-identifiability modulus measuring the worst-case error among joint distributions with population excess risk at most $\varepsilon$. For data with separated modes pinned down by sufficiently large visible contexts, schedules that always retain such contexts can permit nonvanishing mode-weight errors while incurring exponentially small excess risk. An information decomposition explains why: the loss captures mode-weight mismatch only where the visible context leaves the mode uncertain. We prove two-sided bounds showing that, over a fixed range of mode weights, sensitivity to mode reweighting is governed by residual mode uncertainty averaged over the mask schedule. Assigning schedule mass to low-visibility masks that retain this uncertainty yields recovery bounds within the reweighting family. Beyond this family, positive full-mask probability characterizes uniform control of joint KL divergence by excess risk. Exact calculations and controlled gradient optimization validate these predictions.

翻译：暂无翻译