Activation-based steering can personalize large language models at inference time, but its effects in educational settings remain unclear. We study persona vectors for seven character traits in short-answer generation and automated scoring on the ASAP-SAS benchmark across three models spanning two architectures. Persona steering lowers answer quality overall, with much larger effects on open-ended English Language Arts (ELA) prompts than on factual science prompts; interpretive and argumentative tasks are up to 11x more sensitive. On the scoring side, we observe predictable valence-aligned calibration shifts: evil and impolite scorers grade more harshly, while good and optimistic scorers grade more leniently. ELA tasks are 2.5-3x more susceptible to scorer personalization than science tasks, and the Mixture-of-Experts model shows roughly 6x larger calibration shifts than the dense models. To our knowledge, this is the first study to systematically examine the effects of activation-steered persona traits in educational generation and scoring, and the results highlight the need for task-aware and architecture-aware calibration when deploying steered models in educational settings.
翻译:基于激活的引导技术可以在推理时个性化大语言模型,但其在教育场景中的效果尚不明确。我们针对两种架构的三种模型,在ASAP-SAS基准测试中研究了七种性格特征的人格向量在短答案生成与自动评分中的应用。总体而言,人格引导降低了答案质量,对开放型英语语言艺术(ELA)提示的影响远大于事实型科学提示,其中解释性和论证性任务的敏感度高出多达11倍。在评分方面,我们观察到可预测的效价对齐校准偏移:邪恶与不礼貌的评分者评分更严苛,而善良与乐观的评分者评分更宽松。ELA任务对评分者个性化的敏感度是科学任务的2.5-3倍,而混合专家模型(MoE)的校准偏移幅度约是密集模型的6倍。据我们所知,这是首个系统考察激活引导人格特征在教育生成与评分中影响的研究,结果凸显了在教育场景部署引导模型时,需根据具体任务与架构进行校准的必要性。