We investigate the potential of GPT-4~\cite{gpt4} to perform Neural Architecture Search (NAS) -- the task of designing effective neural architectures. Our proposed approach, \textbf{G}PT-4 \textbf{I}nformed \textbf{N}eural \textbf{A}rchitecture \textbf{S}earch (GINAS),leverages the generative capabilities of GPT-4 as a black-box optimiser to quickly navigate the architecture search space, pinpoint promising candidates, and iteratively refine these candidates to improve performance.We assess GINAS across several benchmarks, comparing it with existing state-of-the-art NAS techniques to illustrate its effectiveness. Rather than targeting state-of-the-art performance, our objective is to highlight GPT-4's potential to assist research on a challenging technical problem through a simple prompting scheme that requires relatively limited domain expertise. More broadly, we believe our preliminary results point to future research that harnesses general purpose language models for diverse optimisation tasks. We also highlight important limitations to our study, and note implications for AI safety.

翻译：我们研究了GPT-4~\cite{gpt4}执行神经架构搜索（NAS）——即设计有效神经架构任务的潜力。我们提出的方法——**G**PT-4 **I**nformed **N**eural **A**rchitecture **S**earch（GINAS）——利用GPT-4的生成能力作为黑箱优化器，快速导航架构搜索空间，定位有潜力的候选架构，并通过迭代优化这些候选架构以提升性能。我们在多个基准测试上评估了GINAS，并与现有最先进的NAS技术进行了对比，以展示其有效性。我们的目标并非追求最先进的性能，而是通过一种仅需相对有限领域知识的简单提示方案，凸显GPT-4在辅助解决技术难题方面的潜力。更广泛地，我们认为初步结果表明，未来研究可借助通用语言模型处理多样化优化任务。同时，我们指出了本研究的重要局限性，并探讨了对人工智能安全的影响。