Language models, such as GPT-3.5 and ChatGPT, demonstrate remarkable abilities to follow diverse human instructions and perform a wide range of tasks. However, when probing language models using a range of basic table-understanding tasks, we observe that today's language models are still sub-optimal in many table-related tasks, likely because they are pre-trained predominantly on \emph{one-dimensional} natural-language texts, whereas relational tables are \emph{two-dimensional} objects. In this work, we propose a new "\emph{table-tuning}" paradigm, where we continue to train/fine-tune language models like GPT-3.5 and ChatGPT, using diverse table-tasks synthesized from real tables as training data, with the goal of enhancing language models' ability to understand tables and perform table tasks. We show that our resulting Table-GPT models demonstrate (1) better \emph{table-understanding} capabilities, by consistently outperforming the vanilla GPT-3.5 and ChatGPT, on a wide-range of table tasks, including holdout unseen tasks, and (2) strong \emph{generalizability}, in its ability to respond to diverse human instructions to perform new table-tasks, in a manner similar to GPT-3.5 and ChatGPT.

翻译：语言模型（如GPT-3.5和ChatGPT）展现出遵循多样化人类指令并执行广泛任务的卓越能力。然而，在通过一系列基础表格理解任务测试这些语言模型时，我们发现当前的语言模型在许多表格相关任务中仍表现欠佳，这很可能是因为它们主要在“一维”自然语言文本上进行预训练，而关系型表格是“二维”对象。在本研究中，我们提出了一种新的“表格微调”范式，即使用从真实表格中合成的多样化表格任务作为训练数据，持续训练/微调GPT-3.5和ChatGPT等语言模型，旨在增强语言模型理解表格及执行表格任务的能力。结果表明，我们生成的Table-GPT模型展现出：（1）更强的“表格理解”能力——在包括未见过的保留任务在内的广泛表格任务中，始终优于原始GPT-3.5和ChatGPT；（2）强大的“泛化能力”——能够像GPT-3.5和ChatGPT一样响应多样化的人类指令，执行新的表格任务。