Much of software engineering research focuses on tools, algorithms, and optimization of software. Recently, we, as a community, have come to acknowledge that there is a gap in meta-research and addressing the human-factors in software engineering research. Through meta research, we aim to deepen our understanding of online participant recruitment and human-subjects software engineering research. In this paper we motivate the need to consider the unique challenges that human studies pose in software engineering research. We present several challenges faced by our research team in several distinct research studies, how they affected research, and motivate how, as researchers, we can address these challenges. We present results from a pilot study and categorize issues faced into three broad categories including participant recruitment, community engagement, and data poisoning. We further discuss how we can address these challenges and outline the benefits a full-study could provide to the software engineering research community.

翻译：软件工程研究大多聚焦于工具、算法及软件优化。近年来，学术界逐渐认识到元研究及人因工程在软件工程研究中存在缺失。通过元研究，我们旨在深化对在线参与者招募及以人为研究对象的软件工程研究的理解。本文阐述了考虑人因研究在软件工程领域特有挑战的必要性，呈现了研究团队在多项独立研究中遭遇的若干挑战及其对研究进程的影响，并探讨了研究者应对这些挑战的可行路径。通过一项先导性研究，我们将问题归纳为三大类：参与者招募、社区参与及数据投毒。最后，我们进一步讨论了应对策略，并概述了开展完整研究对软件工程研究共同体的潜在价值。