The advent of Large Language Model-driven tools like ChatGPT offers software engineers an interactive alternative to community question-answering (CQA) platforms like Stack Overflow. While Stack Overflow provides benefits from the accumulated crowd-sourced knowledge, it often suffers from unpleasant comments, reactions, and long waiting times. In this study, we assess the efficacy of ChatGPT in providing solutions to software engineering questions by analyzing its performance specifically against human answers on 2564 Python and JavaScript questions posted between January 2022 and December 2022 in Stack Overflow. We parse the questions and answers from Stack Overflow, then collect the answers to the same questions from ChatGPT through API, and employ four textual and four cognitive metrics to compare the answers generated by ChatGPT with the accepted answers provided by human subject matter experts to find out the potential reasons for which future knowledge seekers may prefer ChatGPT over CQA platforms. Our analysis indicates that ChatGPT's responses are 66\% shorter and share 35\% more words with the questions, showing a 25\% increase in positive sentiment compared to human responses. ChatGPT's answers' accuracy rate is between 71 to 75\%, with a variation in response characteristics between JavaScript and Python. Additionally, our findings suggest a recent 38\% decrease in comment interactions on Stack Overflow, indicating a shift in community engagement patterns. A supplementary survey with 14 Python and JavaScript professionals validated these findings. While ChatGPT offers quicker, more concise responses, the implications of reduced community involvement warrant further investigation.

翻译：以ChatGPT为代表的大语言模型驱动工具的出现，为软件工程师提供了除Stack Overflow等社区问答平台之外的交互式替代方案。尽管Stack Overflow能够提供来自众包知识积累的益处，但其常伴随不友善的评论、负面反馈及较长的等待时间。本研究通过分析ChatGPT在2564条发布于2022年1月至12月期间Stack Overflow平台的Python与JavaScript问题上的表现，特别针对其与人类答案的对比，评估了ChatGPT为软件工程问题提供解决方案的效能。我们解析了Stack Overflow的问题与答案，通过API收集ChatGPT对相同问题的回复，并采用四项文本指标与四项认知指标，将ChatGPT生成的答案与人类领域专家提供的采纳答案进行比较，以探究未来知识寻求者可能更倾向选择ChatGPT而非社区问答平台的潜在原因。分析表明，ChatGPT的回复长度缩短66%，与问题共享的词汇量增加35%，且相较于人类回复，其积极情感表达提升25%。ChatGPT答案的准确率介于71%至75%之间，且在JavaScript与Python问题上的响应特征存在差异。此外，我们的发现显示Stack Overflow近期评论互动量下降38%，表明社区参与模式正在发生转变。一项针对14位Python与JavaScript专业人士的补充调查验证了上述发现。尽管ChatGPT能够提供更快速、更简洁的响应，但社区参与度降低所带来的影响仍需进一步探究。