Third-party annotation is the status quo for labeling text, but egocentric information such as sentiment and belief can at best only be approximated by a third-person proxy. We introduce author labeling, an annotation technique where the writer of the document itself annotates the data at the moment of creation. We collaborate with a commercial chatbot with over 20,000 users to deploy an author labeling annotation system. This system identifies task-relevant queries, generates on-the-fly labeling questions, and records authors' answers in real time. We train and deploy an online-learning model architecture for product recommendation with author-labeled data to improve performance. We train our model to minimize the prediction error on questions generated for a set of predetermined subjective beliefs using author-labeled responses. Our model achieves a 537% improvement in click-through rate compared to an industry advertising baseline running concurrently. We then compare the quality and practicality of author labeling to three traditional annotation approaches for sentiment analysis and find author labeling to be higher quality, faster to acquire, and cheaper. These findings reinforce existing literature that annotations, especially for egocentric and subjective beliefs, are significantly higher quality when labeled by the author rather than a third party. To facilitate broader scientific adoption, we release an author labeling service for the research community at https://academic.echollm.io.

翻译：第三方标注是文本标注的现状，但诸如情感和信念等自我中心信息最多只能通过第三方代理近似获取。我们引入了作者标注技术，即文档撰写者在创作时自行对数据进行标注。我们与一个拥有超过20,000名用户的商业聊天机器人合作，部署了一套作者标注系统。该系统识别任务相关的查询，实时生成标注问题，并记录作者的回答。我们训练并部署了一个在线学习模型架构，利用作者标注数据进行产品推荐以提升性能。我们训练模型以最小化针对一组预定主观信念所生成问题的预测误差，并使用作者标注的响应进行优化。与同期运行的行业广告基线相比，我们的模型实现了点击率537%的提升。随后，我们将作者标注与三种传统情感分析标注方法的质量和实用性进行比较，发现作者标注质量更高、获取速度更快且成本更低。这些发现强化了现有文献的观点：对于自我中心和主观信念的标注，由作者而非第三方进行标注能显著提高质量。为促进更广泛的科学应用，我们在https://academic.echollm.io向研究社区发布了一项作者标注服务。