Studies often aim to reveal how neural representations encode aspects of an observer's environment, such as its contents or structure. These are ``first-order" representations (FORs), because they're ``about" the external world. A less-common target is ``higher-order" representations (HORs), which are ``about" FORs -- their contents, stability, or uncertainty. HORs of uncertainty appear critically involved in adaptive behaviors including learning under uncertainty, influencing learning rates and internal model updating based on environmental feedback. However, HORs about uncertainty are unlikely to be direct ``read-outs" of FOR characteristics, instead reflecting estimation processes which may be lossy, bias-prone, or distortive and which may also incorporate estimates of distributions of uncertainty the observer is likely to experience. While some research has targeted neural representations of ``instantaneously" estimated uncertainty, how the brain represents \textit{distributions} of expected uncertainty remains largely unexplored. Here, we propose a novel reinforcement learning (RL) based generative artificial intelligence (genAI) approach to explore neural representations of uncertainty distributions. We use existing functional magnetic resonance imaging data, where humans learned to `de-noise' their brain states to achieve target neural patterns, to train denoising diffusion genAI models with RL algorithms to learn noise distributions similar to how humans might learn to do the same. We then explore these models' learned noise-distribution HORs compared to control models trained with traditional backpropagation. Results reveal model-dependent differences in noise distribution representations -- with the RL-based model offering much higher explanatory power for human behavior -- offering an exciting path towards using genAI to explore neural noise-distribution HORs.

翻译：研究通常旨在揭示神经表征如何编码观察者环境的各个方面，例如其内容或结构。这些属于"一阶"表征，因为它们"关于"外部世界。一个较少被关注的目标是"高阶"表征，它们"关于"一阶表征——包括其内容、稳定性或不确定性。关于不确定性的高阶表征似乎对适应性行为至关重要，包括不确定性下的学习，影响学习速率和基于环境反馈的内部模型更新。然而，关于不确定性的高阶表征不太可能是一阶表征特征的直接"读出"，而是反映了可能具有信息损失、易产生偏差或扭曲特性的估计过程，并且可能还整合了观察者可能经历的不确定性分布的估计。虽然已有研究针对"瞬时"估计不确定性的神经表征，但大脑如何表征预期不确定性的分布仍然在很大程度上未被探索。在此，我们提出一种基于强化学习的新型生成式人工智能方法，用于探索不确定性分布的神经表征。我们利用现有的功能磁共振成像数据——其中人类学习通过"去噪"其大脑状态以实现目标神经模式——通过强化学习算法训练去噪扩散生成式人工智能模型，使其学习噪声分布的方式类似于人类可能的学习过程。随后，我们比较这些模型学习到的噪声分布高阶表征与使用传统反向传播训练的对照模型。结果显示噪声分布表征存在模型依赖性差异——基于强化学习的模型对人类行为具有更高的解释力——这为利用生成式人工智能探索神经噪声分布的高阶表征开辟了令人兴奋的研究路径。