Software is vital for the advancement of biology and medicine. Analysis of usage and impact metrics can help developers determine user and community engagement, justify additional funding, encourage additional use, identify unanticipated use cases, and help define improvement areas. However, there are challenges associated with these analyses including distorted or misleading metrics, as well as ethical and security concerns. More attention to the nuances involved in capturing impact across the spectrum of biological software is needed. Furthermore, some tools may be especially beneficial to a small audience, yet may not have compelling typical usage metrics. We propose more general guidelines, as well as strategies for more specific types of software. We highlight outstanding issues regarding how communities measure or evaluate software impact. To get a deeper understanding of current practices for software evaluations, we performed a survey of participants in the Informatics Technology for Cancer Research (ITCR) program funded by the National Cancer Institute (NCI). We also investigated software among this community and others to assess how often infrastructure that supports such evaluations is implemented and how this impacts rates of papers describing usage of the software. We find that developers recognize the utility of analyzing software usage, but struggle to find the time or funding for such analyses. We also find that infrastructure such as social media presence, more in-depth documentation, the presence of software health metrics, and clear information on how to contact developers seem to be associated with increased usage rates. Our findings can help scientific software developers make the most out of evaluations of their software.

翻译：软件对于生物学和医学的发展至关重要。使用情况和影响力指标的分析可帮助开发者确定用户和社区参与度、论证额外资助的合理性、鼓励更广泛的使用、识别非预期的应用场景，并明确改进方向。然而，此类分析面临挑战，包括指标失真或误导性解读，以及伦理与安全隐患。我们需要更关注生物软件从整体到局部的影响力捕捉所涉及的细微差别。此外，某些工具可能对特定小规模受众极具价值，但其常规使用指标未必显著。我们提出了通用性更强的指导原则，以及针对特定类型软件的策略。我们着重讨论了当前各社区衡量或评估软件影响力时存在的突出问题。为深入理解软件评估的现行实践，我们对美国国家癌症研究所（NCI）资助的“癌症研究信息学技术（ITCR）”计划参与者进行了调查。我们还分析了该社区及其他社区的软件，以评估支撑此类评估的基础设施建设频率，以及这对描述软件使用情况的论文发表率的影响。研究发现，开发者认可分析软件使用的价值，但难以为此类分析投入时间或获取资助。同时，社交媒体存在、更详尽的文档、软件健康指标的可视化、以及联系开发者的明确指引等基础设施，似乎与更高的使用率相关。我们的发现可帮助科学软件开发者最大程度地利用对其软件的评估。