Social science NLP tasks, such as emotion or humor detection, are required to capture the semantics along with the implicit pragmatics from text, often with limited amounts of training data. Instruction tuning has been shown to improve the many capabilities of large language models (LLMs) such as commonsense reasoning, reading comprehension, and computer programming. However, little is known about the effectiveness of instruction tuning on the social domain where implicit pragmatic cues are often needed to be captured. We explore the use of instruction tuning for social science NLP tasks and introduce Socialite-Llama -- an open-source, instruction-tuned Llama. On a suite of 20 social science tasks, Socialite-Llama improves upon the performance of Llama as well as matches or improves upon the performance of a state-of-the-art, multi-task finetuned model on a majority of them. Further, Socialite-Llama also leads to improvement on 5 out of 6 related social tasks as compared to Llama, suggesting instruction tuning can lead to generalized social understanding. All resources including our code, model and dataset can be found through bit.ly/socialitellama.

翻译：社会科学领域的自然语言处理任务（如情感或幽默检测）需要从文本中捕获语义信息及隐式语用特征，且通常缺乏充足的训练数据。指令微调已被证明能提升大语言模型（LLM）在常识推理、阅读理解、计算机编程等多方面的能力。然而，在需要捕捉隐式语用线索的社会科学领域，指令微调的效用尚不明确。我们探索了指令微调在社会科学NLP任务中的应用，并提出了Socialite-Llama——一个开源、经指令微调的Llama模型。在包含20项社会科学任务的测试集上，Socialite-Llama不仅提升了Llama的性能，还在多数任务上达到或超越了当前最优的多任务微调模型。此外，与Llama相比，Socialite-Llama在6项相关社会任务中的5项上取得改进，表明指令微调可促进通用社会理解能力。所有资源（包括代码、模型和数据集）可通过bit.ly/socialitellama获取。