Recent breakthroughs in natural language processing (NLP) have permitted the synthesis and comprehension of coherent text in an open-ended way, therefore translating the theoretical algorithms into practical applications. The large language-model (LLM) has significantly impacted businesses such as report summarization softwares and copywriters. Observations indicate, however, that LLMs may exhibit social prejudice and toxicity, posing ethical and societal dangers of consequences resulting from irresponsibility. Large-scale benchmarks for accountable LLMs should consequently be developed. Although several empirical investigations reveal the existence of a few ethical difficulties in advanced LLMs, there is no systematic examination and user study of the ethics of current LLMs use. To further educate future efforts on constructing ethical LLMs responsibly, we perform a qualitative research method on OpenAI's ChatGPT to better understand the practical features of ethical dangers in recent LLMs. We analyze ChatGPT comprehensively from four perspectives: 1) \textit{Bias} 2) \textit{Reliability} 3) \textit{Robustness} 4) \textit{Toxicity}. In accordance with our stated viewpoints, we empirically benchmark ChatGPT on multiple sample datasets. We find that a significant number of ethical risks cannot be addressed by existing benchmarks, and hence illustrate them via additional case studies. In addition, we examine the implications of our findings on the AI ethics of ChatGPT, as well as future problems and practical design considerations for LLMs. We believe that our findings may give light on future efforts to determine and mitigate the ethical hazards posed by machines in LLM applications.

翻译：自然语言处理领域的最新突破使得能够以开放式方式生成和理解连贯文本，从而将理论算法转化为实际应用。大型语言模型已显著影响报告摘要软件和文案撰写等业务。然而，观察表明，大型语言模型可能表现出社会偏见与毒性，带来因不负责任行为而引发的伦理与社会风险。因此，亟需开发针对负责任大型语言模型的大规模基准测试。尽管若干实证研究揭示了高级大型语言模型中存在少量伦理问题，但针对当前大型语言模型使用伦理的系统性检验与用户研究仍付之阙如。为更负责任地指导未来构建伦理大型语言模型的工作，我们采用质性研究方法对OpenAI的ChatGPT进行深入分析，以理解近期大型语言模型中伦理风险的实际特征。我们从四个维度全面剖析ChatGPT：1）偏见 2）可靠性 3）鲁棒性 4）毒性。基于上述研究视角，我们在多个样本数据集上对ChatGPT进行实证基准测试。研究发现，大量伦理风险无法通过现有基准测试加以解决，因而通过额外案例研究对其进行阐释。此外，我们探讨了研究发现对ChatGPT AI伦理的影响，以及大型语言模型面临的未来挑战与实践设计考量。我们相信，本研究的发现将为未来在大型语言模型应用中识别并减轻机器引发的伦理危害提供启示。