Large Language Models(LLMs) have been attracting attention due to a ability called in-context learning(ICL). ICL, without updating the parameters of a LLM, it is possible to achieve highly accurate inference based on rules ``in the context'' by merely inputting a training data into the prompt. Although ICL is a developing field with many unanswered questions, LLMs themselves serves as a inference model, seemingly realizing inference without explicitly indicate ``inductive bias''. On the other hand, a code generation is also a highlighted application of LLMs. The accuracy of code generation has dramatically improved, enabling even non-engineers to generate code to perform the desired tasks by crafting appropriate prompts. In this paper, we propose a novel ``learning'' method called an ``Inductive-Bias Learning (IBL)'', which combines the techniques of ICL and code generation. An idea of IBL is straightforward. Like ICL, IBL inputs a training data into the prompt and outputs a code with a necessary structure for inference (we referred to as ``Code Model'') from a ``contextual understanding''. Despite being a seemingly simple approach, IBL encompasses both a ``property of inference without explicit inductive bias'' inherent in ICL and a ``readability and explainability'' of the code generation. Surprisingly, generated Code Models have been found to achieve predictive accuracy comparable to, and in some cases surpassing, ICL and representative machine learning models. Our IBL code is open source: https://github.com/fuyu-quant/IBLM

翻译：大型语言模型（LLMs）因其被称为上下文学习（ICL）的能力而备受关注。通过ICL，无需更新LLM的参数，仅将训练数据输入提示中，即可基于“上下文”中的规则实现高精度推理。尽管ICL是一个充满未解之谜的新兴领域，但LLM本身充当推理模型，似乎在无需明确指定“归纳偏置”的情况下实现了推理。另一方面，代码生成也是LLM的一个突出应用。代码生成的准确性已显著提升，即使是非工程师也能通过精心设计提示，生成执行所需任务的代码。本文提出了一种新颖的“学习”方法，称为“归纳偏置学习（IBL）”，它融合了ICL和代码生成的技术。IBL的思想直截了当：如同ICL一样，IBL将训练数据输入提示中，并通过“上下文理解”输出具有推理所需结构的代码（我们称之为“代码模型”）。尽管这种方法看似简单，但IBL同时包含了ICL固有的“无需显式归纳偏置的推理特性”和代码生成的“可读性与可解释性”。令人惊讶的是，生成的代码模型在预测准确性上已达到与ICL及代表性机器学习模型相当甚至更高的水平。我们的IBL代码已开源：https://github.com/fuyu-quant/IBLM