Simulating sampling algorithms with people has proven a useful method for efficiently probing and understanding their mental representations. We propose that the same methods can be used to study the representations of Large Language Models (LLMs). While one can always directly prompt either humans or LLMs to disclose their mental representations introspectively, we show that increased efficiency can be achieved by using LLMs as elements of a sampling algorithm. We explore the extent to which we recover human-like representations when LLMs are interrogated with Direct Sampling and Markov chain Monte Carlo (MCMC). We found a significant increase in efficiency and performance using adaptive sampling algorithms based on MCMC. We also highlight the potential of our method to yield a more general method of conducting Bayesian inference \textit{with} LLMs.

翻译：通过使用人类模拟采样算法已被证明是一种有效探测和理解其心理表征的方法。我们提出，同样的方法可用于研究大型语言模型（LLMs）的表征。虽然人们可以直接提示人类或LLMs通过内省方式披露其心理表征，但我们表明，将LLMs作为采样算法的组成部分可以实现更高的效率。我们探究了在直接采样和马尔可夫链蒙特卡洛（MCMC）方法下，使用LLMs进行提问时能在多大程度上恢复类人表征。我们发现，基于MCMC的自适应采样算法在效率和性能上均有显著提升。我们还强调了该方法在利用LLMs进行贝叶斯推断方面具有更普适性的潜力。