Despite advances in language modelling, distributional methods that build semantic representations from co-occurrences fail to discriminate between plausible and implausible events. In this work, we investigate how plausibility prediction can be improved by injecting latent knowledge prompted from large language models using parameter-efficient fine-tuning. We train 12 task adapters to learn various physical properties and association measures and perform adapter fusion to compose latent semantic knowledge from each task on top of pre-trained AlBERT embeddings. We automate auxiliary task data generation, which enables us to scale our approach and fine-tune our learned representations across two plausibility datasets. Our code is available at https://github.com/Jacob-Chmura/plausibility-vaccine.

翻译：尽管语言建模技术取得了进展，但基于共现构建语义表征的分布式方法仍无法有效区分可信与不可信事件。本研究探讨如何通过参数高效微调，注入从大型语言模型中提取的潜在知识以改进可信度预测。我们训练了12个任务适配器来学习多种物理属性和关联度量，并在预训练的AlBERT嵌入基础上通过适配器融合技术组合各任务的潜在语义知识。我们实现了辅助任务数据的自动化生成，这使得我们能够扩展该方法并在两个可信度数据集上微调学习到的表征。相关代码已发布于https://github.com/Jacob-Chmura/plausibility-vaccine。