Designing machine intelligence to converse with a human user necessarily requires an understanding of how humans participate in conversation, and thus conversation modeling is an important task in natural language processing. New breakthroughs in architecture and data gathering continue to push the performance of such conversational AI models. However, designs neglect the gradual buildup in sentence structure and complexity experienced by humans as we learn to communicate. During training, our model accepts one or more sentences as input and attempts to predict the next sentence in the conversation one word at a time, so our goal is to separate training into segments, with each segment's corpus comprised of longer sentence pairs than the previous one. This will mimic the desired "buildup" component of human learning. We begin with only "short" length sentence pairs, then only "medium" length pairs, and so on. A majority of our experiments were toward optimizing this technique, ensuring a proper representation of the technique's potential, since many of the details were new questions. Our segment-trained models were then able to achieve lower validation loss at the end of training than models trained with standard text preparation. This segmented training is straightforward to implement and our results provide a general direction for future research to implement and improve it.

翻译：设计能够与人类用户进行对话的机器智能必然需要理解人类如何参与对话，因此对话建模是自然语言处理中的重要任务。架构与数据收集方面的新突破持续推动着此类对话AI模型的性能提升。然而，现有设计忽略了人类在学习交流过程中句子结构与复杂性的渐进式积累。在训练过程中，我们的模型接受一个或多个句子作为输入，并尝试逐词预测对话中的下一个句子，因此我们的目标是将训练分为多个阶段，每个阶段的语料库包含比前一阶段更长的句子对。这将模拟人类学习所需的“逐步积累”要素。我们首先仅使用“短句”长度句子对，随后仅使用“中等长度”句子对，依此类推。我们的大部分实验致力于优化这一技术，确保充分展现该技术的潜力，因为许多细节均是全新的问题。最终，采用分阶段训练的模型在训练结束时获得的验证损失低于使用标准文本准备的模型。这种分阶段训练实现简单，我们的结果为未来研究提供了实施与改进该方法的总体方向。