The problem of natural language generation, and, more specifically, method name prediction, faces significant difficulties when proposed models need to be evaluated on test data. Such a metric would need to consider the versatility with which a single method can be named, with respect to both semantics and syntax. Measuring the direct overlap between the predicted and reference (true) sequences will not be able to capture these subtleties. Other existing embedding based metrics either do not measure precision and recall or impose strict unrealistic assumptions on both sequences. To address these issues, we propose a new metric that, on the one hand, is very simple and lightweight, and, on the other hand, is able to calculate precision and recall without resorting to any assumptions while obtaining good performance with respect to the human judgement.

翻译：自然语言生成问题，特别是方法名预测问题，在需要基于测试数据评估所提出模型时面临显著困难。此类度量标准需考虑单个方法在语义和句法层面命名方式的多样性。直接测量预测序列与参考（真实）序列间的重叠度将无法捕捉这些细微差别。其他现有的基于嵌入的度量方法要么无法测量精确率和召回率，要么对两个序列施加严格且不切实际的假设。为解决这些问题，我们提出了一种新度量标准，该标准一方面非常简单轻量，另一方面能够在不依赖任何假设的情况下计算精确率和召回率，同时在人类判断相关性方面获得良好性能。