Q&A platforms have been an integral part of the web-help-seeking behavior of programmers over the past decade. However, with the recent introduction of ChatGPT, the paradigm of web-help-seeking behavior is experiencing a shift. Despite the popularity of ChatGPT, no comprehensive study has been conducted to evaluate the characteristics or usability of ChatGPT's answers to software engineering questions. To bridge the gap, we conducted the first in-depth analysis of ChatGPT's answers to 517 Stack Overflow (SO) questions and examined the correctness, consistency, comprehensiveness, and conciseness of ChatGPT's answers. Furthermore, we conducted a large-scale linguistic analysis, and a user study to understand the characteristics of ChatGPT answers from linguistic and human aspects. Our analysis shows that 52% of ChatGPT answers are incorrect and 77% are verbose. Nonetheless, ChatGPT answers are still preferred 39.34% of the time due to their comprehensiveness and well-articulated language style. Our result implies the necessity of close examination and rectification of errors in ChatGPT, at the same time creating awareness among its users of the risks associated with seemingly correct ChatGPT answers.

翻译：问答平台在过去十年中一直是程序员网络求助行为的重要组成部分。然而，随着ChatGPT的最近引入，网络求助行为的范式正在经历转变。尽管ChatGPT广受欢迎，但尚无全面研究评估ChatGPT对软件工程问题回答的特性或可用性。为填补这一空白，我们首次对ChatGPT针对517个Stack Overflow（SO）问题的回答进行了深度分析，考察了ChatGPT回答的正确性、一致性、全面性和简洁性。此外，我们进行了大规模的语言学分析和用户研究，以从语言和人类层面理解ChatGPT回答的特征。我们的分析显示，52%的ChatGPT回答存在错误，77%的回答冗长。尽管如此，由于其全面性和表述清晰的语言风格，ChatGPT的回答仍有39.34%的时间被用户偏好。我们的结果暗示了仔细检查和修正ChatGPT错误的必要性，同时提醒用户注意看似正确的ChatGPT回答所伴随的风险。