We present SAINE, an Scientific Annotation and Inference ENgine based on a set of standard open-source software, such as Label Studio and MLflow. We show that our annotation engine can benefit the further development of a more accurate classification. Based on our previous work on hierarchical discipline classifications, we demonstrate its application using SAINE in understanding the space for scholarly publications. The user study of our annotation results shows that user input collected with the help of our system can help us better understand the classification process. We believe that our work will help to foster greater transparency and better understand scientific research. Our annotation and inference engine can further support the downstream meta-science projects. We welcome collaboration and feedback from the scientific community on these projects. The demonstration video can be accessed from https://youtu.be/yToO-G9YQK4. A live demo website is available at https://app.heartex.com/user/signup/?token=e2435a2f97449fa1 upon free registration.
翻译:我们提出SAINE——一个基于Label Studio、MLflow等标准开源软件的科研标注与推理引擎。研究表明,该标注引擎能推动更精确分类方法的后续发展。基于我们在学科层级分类方面的前期工作,我们以SAINE为例演示其在探索学术出版空间中的应用。用户对标注结果的研究表明,借助本系统收集的用户输入有助于我们更深入地理解分类过程。我们认为本工作将促进科研透明度提升与科研认知深化。该标注与推理引擎可进一步支持下游元科学研究项目。我们欢迎科学界就这些项目开展合作与反馈。演示视频请访问https://youtu.be/yToO-G9YQK4,实时演示网站可通过免费注册获取:https://app.heartex.com/user/signup/?token=e2435a2f97449fa1。