The Chinese character riddle is a unique form of cultural entertainment specific to the Chinese language. It typically comprises two parts: the riddle description and the solution. The solution to the riddle is a single character, while the riddle description primarily describes the glyph of the solution, occasionally supplemented with its explanation and pronunciation. Solving Chinese character riddles is a challenging task that demands understanding of character glyph, general knowledge, and a grasp of figurative language. In this paper, we construct a \textbf{C}hinese \textbf{C}haracter riddle dataset named CC-Riddle, which covers the majority of common simplified Chinese characters. The construction process is a combination of web crawling, language model generation and manual filtering. In generation stage, we input the Chinese phonetic alphabet, glyph and meaning of the solution character into the generation model, which then produces multiple riddle descriptions. The generated riddles are then manually filtered and the final CC-Riddle dataset is composed of both human-written riddles and these filtered, generated riddles. In order to assess the performance of language models on the task of solving character riddles, we use retrieval-based, generative and multiple-choice QA strategies to test three language models: BERT, ChatGPT and ChatGLM. The test results reveal that current language models still struggle to solve Chinese character riddles. CC-Riddle is publicly available at \url{https://github.com/pku0xff/CC-Riddle}.

翻译：汉字字谜是汉语特有的一种文化娱乐形式。它通常由谜面和谜底两部分组成：谜底为一个汉字，而谜面主要描述谜底的形态，有时辅以其释义和读音。解答汉字字谜是一项具有挑战性的任务，需要理解字形、掌握常识以及领会比喻性语言。本文构建了一个名为CC-Riddle的汉字字谜数据集，覆盖了大多数常见简体汉字。数据集的构建过程结合了网络爬取、语言模型生成和人工筛选。在生成阶段，我们将谜底汉字的拼音、字形和语义输入生成模型，模型据此生成多个谜面描述。生成的谜面随后经过人工筛选，最终CC-Riddle数据集由人工编写的谜面和筛选后的生成谜面共同组成。为评估语言模型在解答字谜任务上的性能，我们采用基于检索、生成式和多选题问答三种策略，对BERT、ChatGPT和ChatGLM三个语言模型进行了测试。测试结果表明，当前语言模型在解答汉字字谜方面仍存在困难。CC-Riddle数据集已在\url{https://github.com/pku0xff/CC-Riddle}公开发布。