Large Language Models (LLMs) have demonstrated remarkable success in various natural language processing and software engineering tasks, such as code generation. The LLMs are mainly utilized in the prompt-based zero/few-shot paradigm to guide the model in accomplishing the task. GPT-based models are one of the popular ones studied for tasks such as code comment generation or test generation. These tasks are `generative' tasks. However, there is limited research on the usage of LLMs for `non-generative' tasks such as classification using the prompt-based paradigm. In this preliminary exploratory study, we investigated the applicability of LLMs for Code Clone Detection (CCD), a non-generative task. By building a mono-lingual and cross-lingual CCD dataset derived from CodeNet, we first investigated two different prompts using ChatGPT to detect Type-4 code clones in Java-Java and Java-Ruby pairs in a zero-shot setting. We then conducted an analysis to understand the strengths and weaknesses of ChatGPT in CCD. ChatGPT surpasses the baselines in cross-language CCD attaining an F1-score of 0.877 and achieves comparable performance to fully fine-tuned models for mono-lingual CCD, with an F1-score of 0.878. Also, the prompt and the difficulty level of the problems has an impact on the performance of ChatGPT. Finally we provide insights and future directions based on our initial analysis

翻译：大型语言模型（LLMs）在自然语言处理和软件工程任务（如代码生成）中展现出显著的成功。LLMs 主要基于提示驱动的零样本/少样本范式，以引导模型完成任务。以GPT为基础的模型是学术界重点研究的对象之一，常用于代码注释生成或测试生成等任务。这些任务属于“生成型”任务。然而，基于提示范式将LLMs用于分类等“非生成型”任务的研究尚不充分。在本项初步探索性研究中，我们考察了LLMs应用于代码克隆检测（CCD）这一非生成型任务的可行性。通过构建源自CodeNet的单语和跨语言CCD数据集，我们首先设计了两种不同提示，使用ChatGPT在零样本场景下检测Java-Java与Java-Ruby代码对中的第4类代码克隆。随后，我们分析并总结了ChatGPT在CCD任务中的优势与不足。ChatGPT在跨语言CCD任务中超越基线，F1分数达0.877；在单语CCD任务中，其性能与完全微调的模型相当，F1分数为0.878。此外，提示方式及问题的难度水平对ChatGPT的表现有显著影响。最后，基于初步分析结果，我们提出了相关见解与未来研究方向。