There has been a lot of work in question generation where different methods to provide target answers as input, have been employed. This experimentation has been mostly carried out for RNN based models. We use three different methods and their combinations for incorporating answer information and explore their effect on several automatic evaluation metrics. The methods that are used are answer prompting, using a custom product method using answer embeddings and encoder outputs, choosing sentences from the input paragraph that have answer related information, and using a separate cross-attention attention block in the decoder which attends to the answer. We observe that answer prompting without any additional modes obtains the best scores across rouge, meteor scores. Additionally, we use a custom metric to calculate how many of the generated questions have the same answer, as the answer which is used to generate them.
翻译:在问答生成领域,已有大量工作采用不同方法将目标答案作为输入信息。这些实验主要基于循环神经网络模型展开。本研究采用三种方法及其组合来整合答案信息,并探究其在多种自动评估指标上的影响。所使用的方法包括:答案提示法、通过答案嵌入与编码器输出构建的自定义乘积方法、从输入段落中提取包含答案相关信息的句子,以及在解码器中引入独立交叉注意力模块以关注答案信息。实验结果表明,在不使用额外模式的情况下,仅采用答案提示法在ROUGE和METEOR评分上取得最佳效果。此外,我们采用自定义评估指标,统计生成问题中答案与用于生成问题的原始答案保持一致的比例。