The meanings of words and phrases depend not only on where they are used (contexts) but also on who use them (writers). Pretrained language models (PLMs) are powerful tools for capturing context, but they are typically pretrained and fine-tuned for universal use across different writers. This study aims to improve the accuracy of text understanding tasks by personalizing the fine-tuning of PLMs for specific writers. We focus on a general setting where only the plain text from target writers are available for personalization. To avoid the cost of fine-tuning and storing multiple copies of PLMs for different users, we exhaustively explore using writer-specific prompts to personalize a unified PLM. Since the design and evaluation of these prompts is an underdeveloped area, we introduce and compare different types of prompts that are possible in our setting. To maximize the potential of prompt-based personalized fine-tuning, we propose a personalized intermediate learning based on masked language modeling to extract task-independent traits of writers' text. Our experiments, using multiple tasks, datasets, and PLMs, reveal the nature of different prompts and the effectiveness of our intermediate learning approach.

翻译：词语和短语的含义不仅取决于其使用的环境（上下文），还取决于使用它们的人（作者）。预训练语言模型（PLMs）是捕获上下文的强大工具，但它们通常在不同作者间进行通用预训练和微调。本研究旨在通过针对特定作者个性化微调PLMs，提升文本理解任务的准确性。我们聚焦于一种通用设置：仅使用目标作者的纯文本进行个性化。为避免为不同用户微调并存储多个PLMs副本的成本，我们深入探索利用作者特定提示（writer-specific prompts）来个性化统一PLM。由于此类提示的设计与评估仍属未充分开发领域，我们引入并比较了本设置中可能采用的多种提示类型。为充分发挥基于提示的个性化微调潜力，我们提出基于掩码语言建模的个性化中间学习（personalized intermediate learning），以提取作者文本中与任务无关的特征。通过多项任务、数据集及PLMs的实验，我们揭示了不同提示的本质特性及所提中间学习方法的有效性。