Decoder-only large language models are increasingly used as behavioral encoders for user representation learning, yet the impact of attention masking on the quality of user embeddings remains underexplored. In this work, we conduct a systematic study of causal, hybrid, and bidirectional attention masks within a unified contrastive learning framework trained on large-scale real-world Alipay data that integrates long-horizon heterogeneous user behaviors. To improve training dynamics when transitioning from causal to bidirectional attention, we propose Gradient-Guided Soft Masking, a gradient-based pre-warmup applied before a linear scheduler that gradually opens future attention during optimization. Evaluated on 9 industrial user cognition benchmarks covering prediction, preference, and marketing sensitivity tasks, our approach consistently yields more stable training and higher-quality bidirectional representations compared with causal, hybrid, and scheduler-only baselines, while remaining compatible with decoder pretraining. Overall, our findings highlight the importance of masking design and training transition in adapting decoder-only LLMs for effective user representation learning. Our code is available at https://github.com/JhCircle/Deepfind-GGSM.

翻译：仅解码器大语言模型越来越多地被用作行为编码器以进行用户表征学习，然而注意力掩码对用户嵌入质量的影响仍未得到充分探索。在本工作中，我们在一个统一的对比学习框架内，对因果、混合及双向注意力掩码进行了系统研究，该框架基于整合了长周期异构用户行为的大规模真实世界支付宝数据进行训练。为了改善从因果注意力向双向注意力过渡时的训练动态，我们提出了梯度引导软掩码，这是一种基于梯度的预热方法，在优化过程中通过线性调度器逐步开放未来注意力之前应用。在涵盖预测、偏好和营销敏感性任务的9个工业用户认知基准测试中，与因果、混合及仅使用调度器的基线方法相比，我们的方法始终能产生更稳定的训练和更高质量的双向表征，同时保持与解码器预训练的兼容性。总体而言，我们的研究结果突显了掩码设计和训练过渡在适配仅解码器大语言模型以实现有效用户表征学习中的重要性。我们的代码可在 https://github.com/JhCircle/Deepfind-GGSM 获取。