Enhancing word usage is a desired feature for writing assistance. To further advance research in this area, this paper introduces "Smart Word Suggestions" (SWS) task and benchmark. Unlike other works, SWS emphasizes end-to-end evaluation and presents a more realistic writing assistance scenario. This task involves identifying words or phrases that require improvement and providing substitution suggestions. The benchmark includes human-labeled data for testing, a large distantly supervised dataset for training, and the framework for evaluation. The test data includes 1,000 sentences written by English learners, accompanied by over 16,000 substitution suggestions annotated by 10 native speakers. The training dataset comprises over 3.7 million sentences and 12.7 million suggestions generated through rules. Our experiments with seven baselines demonstrate that SWS is a challenging task. Based on experimental analysis, we suggest potential directions for future research on SWS. The dataset and related codes is available at https://github.com/microsoft/SmartWordSuggestions.

翻译：提升词语使用是写作辅助中一项理想功能。为进一步推动该领域研究，本文提出了“智能词语建议”（Smart Word Suggestions, SWS）任务及基准测试。与其他工作不同，SWS强调端到端评估，并呈现了更贴近实际的写作辅助场景。该任务涉及识别需要改进的词语或短语，并提供替换建议。基准测试包括用于测试的人工标注数据、用于训练的大规模远程监督数据集，以及评估框架。测试数据包含1,000句由英语学习者撰写的句子，配有10位母语者标注的超过16,000条替换建议。训练数据集包含超过370万条句子和1270万条通过规则生成的建议。我们基于七种基线模型的实验表明，SWS是一项具有挑战性的任务。根据实验分析，我们提出了SWS未来研究的潜在方向。数据集及相关代码可从https://github.com/microsoft/SmartWordSuggestions获取。