Much of software engineering research focuses on tools, algorithms, and optimization of software. Recently, we, as a community, have come to acknowledge that there is a gap in meta-research and addressing the human-factors in software engineering research. Through meta research, we aim to deepen our understanding of online participant recruitment and human-subjects software engineering research. In this paper we motivate the need to consider the unique challenges that human studies pose in software engineering research. We present several challenges faced by our research team in several distinct research studies, how they affected research, and motivate how, as researchers, we can address these challenges. We present results from a pilot study and categorize issues faced into three broad categories including participant recruitment, community engagement, and data poisoning. We further discuss how we can address these challenges and outline the benefits a full-study could provide to the software engineering research community.

翻译：软件工程研究大多聚焦于工具、算法及软件优化。近期，我们学术界开始认识到，在元研究以及解决软件工程研究中的人因因素方面存在空白。通过元研究，我们旨在深化对在线参与者招募及以人类为受试对象的软件工程研究的理解。本文阐述了考虑人类研究在软件工程研究中所带来的独特挑战的必要性。我们介绍了研究团队在不同研究项目中面临的若干挑战及其对研究进程的影响，并探讨了作为研究者应如何应对这些挑战。我们呈现了一项试点研究的结果，将遇到的问题归纳为三大类：参与者招募、社群参与及数据投毒。我们进一步讨论了应对这些挑战的策略，并概述了完整研究可为软件工程研究社区带来的益处。