A good theory of mathematical beauty is more practical than any current observation, as new predictions of physical reality can be verified self-consistently. This belief applies to the current status of understanding deep neural networks including large language models and even the biological intelligence. Toy models provide a metaphor of physical reality, allowing mathematically formulating that reality (i.e., the so-called theory), which can be updated as more conjectures are justified or refuted. One does not need to pack all details into a model, but rather, more abstract models are constructed, as complex systems like brains or deep networks have many sloppy dimensions but much less stiff dimensions that strongly impact macroscopic observables. This kind of bottom-up mechanistic modeling is still promising in the modern era of understanding the natural or artificial intelligence. Here, we shed light on eight challenges in developing theory of intelligence following this theoretical paradigm.

翻译：数学之美的一个优秀理论比任何当前观测更为实用，因为对物理现实的新预测可以通过自洽验证。这一信念适用于当前对深度神经网络（包括大语言模型乃至生物智能）的理解现状。简化模型为物理现实提供了隐喻，使人们能够以数学形式表述这一现实（即所谓的理论），而该理论可随着更多猜想得到证实或反驳而更新。人们无需将所有细节纳入模型，相反，应构建更抽象的模型，因为像大脑或深度网络这样的复杂系统存在大量冗余维度，但影响宏观可观测量的刚性维度却少得多。这种自下而上的机理建模在当今理解自然或人工智能的时代仍颇具前景。在此，我们阐述了遵循这一理论范式发展智能理论所面临的八大挑战。