The evaluation of large language models is an essential task in the field of language understanding and generation. As language models continue to advance, the need for effective benchmarks to assess their performance has become imperative. In the context of Traditional Chinese, there is a scarcity of comprehensive and diverse benchmarks to evaluate the capabilities of language models, despite the existence of certain benchmarks such as DRCD, TTQA, CMDQA, and FGC dataset. To address this gap, we propose a novel set of benchmarks that leverage existing English datasets and are tailored to evaluate language models in Traditional Chinese. These benchmarks encompass a wide range of tasks, including contextual question-answering, summarization, classification, and table understanding. The proposed benchmarks offer a comprehensive evaluation framework, enabling the assessment of language models' capabilities across different tasks. In this paper, we evaluate the performance of GPT-3.5, Taiwan-LLaMa-v1.0, and Model 7-C, our proprietary model, on these benchmarks. The evaluation results highlight that our model, Model 7-C, achieves performance comparable to GPT-3.5 with respect to a part of the evaluated capabilities. In an effort to advance the evaluation of language models in Traditional Chinese and stimulate further research in this field, we have open-sourced our benchmark and opened the model for trial.

翻译：大型语言模型的评估是语言理解与生成领域的一项关键任务。随着语言模型的持续演进，亟需有效的基准测试来评估其性能。在传统中文的语境下，尽管存在如DRCD、TTQA、CMDQA和FGC数据集等部分基准，但全面且多样化的评估基准仍较为稀缺。为弥补这一不足，我们提出了一套新颖的基准测试方案，该方案利用现有的英文数据集，并针对传统中文语言模型评估进行定制。这些基准涵盖广泛的任务类型，包括上下文问答、摘要生成、文本分类以及表格理解。所提出的基准提供了一个全面的评估框架，能够跨不同任务评估语言模型的能力。本文中，我们分别评估了GPT-3.5、Taiwan-LLaMa-v1.0以及我们的专有模型Model 7-C在这些基准上的表现。评估结果表明，在部分评估能力上，我们的Model 7-C模型取得了与GPT-3.5相当的性能。为推动传统中文语言模型的评估进展并激发该领域的进一步研究，我们已开源了该基准测试，并开放了模型供试用。