This paper presents a novel sampling scheme for masked non-autoregressive generative modeling. We identify the limitations of TimeVQVAE, MaskGIT, and Token-Critic in their sampling processes, and propose Enhanced Sampling Scheme (ESS) to overcome these limitations. ESS explicitly ensures both sample diversity and fidelity, and consists of three stages: Naive Iterative Decoding, Critical Reverse Sampling, and Critical Resampling. ESS starts by sampling a token set using the naive iterative decoding as proposed in MaskGIT, ensuring sample diversity. Then, the token set undergoes the critical reverse sampling, masking tokens leading to unrealistic samples. After that, critical resampling reconstructs masked tokens until the final sampling step is reached to ensure high fidelity. Critical resampling uses confidence scores obtained from a self-Token-Critic to better measure the realism of sampled tokens, while critical reverse sampling uses the structure of the quantized latent vector space to discover unrealistic sample paths. We demonstrate significant performance gains of ESS in both unconditional sampling and class-conditional sampling using all the 128 datasets in the UCR Time Series archive.

翻译：本文提出了一种用于掩码非自回归生成建模的新型采样方案。我们识别了TimeVQVAE、MaskGIT和Token-Critic在采样过程中的局限性，并提出了增强采样方案（ESS）来克服这些局限。ESS显式地保证了样本多样性和保真度，由三个阶段组成：朴素迭代解码、关键反向采样和关键重采样。ESS首先使用MaskGIT中提出的朴素迭代方法对标记集进行采样，确保样本多样性。随后，该标记集经历关键反向采样阶段，掩码导致不真实样本的标记。之后，关键重采样重构被掩码的标记，直至达到最终采样步骤，以确保高保真度。关键重采样利用从自Token-Critic获得的置信分数来更好地衡量采样标记的真实性，而关键反向采样则利用量化潜向量空间的结构来发现不真实的样本路径。我们通过在UCR时间序列存档中的全部128个数据集上进行无条件采样和类条件采样，证明了ESS显著的性能提升。