Hyperparameter optimization (HPO) plays a central role in the performance of deep learning models, yet remains computationally expensive and difficult to interpret, particularly for time-series forecasting. While Bayesian Optimization (BO) is a standard approach, it typically treats tuning tasks independently and provides limited insight into its decisions. Recent advances in large language models (LLMs) offer new opportunities to incorporate structured prior knowledge and reasoning into optimization pipelines. We introduce LLM-AutoOpt, a hybrid HPO framework that combines BO with LLM-based contextual reasoning. The framework encodes dataset meta-features, model descriptions, historical optimization outcomes, and target objectives as structured meta-knowledge within LLM prompts, using BO to initialize the search and mitigate cold-start effects. This design enables context-aware and stable hyperparameter refinement while exposing the reasoning behind optimization decisions. Experiments on a multivariate time series forecasting benchmark demonstrate that LLM-AutoOpt achieves improved predictive performance and more interpretable optimization behavior compared to BO and LLM baselines without meta-knowledge.

翻译：超参数优化（HPO）对深度学习模型的性能起着核心作用，但其计算成本高昂且难以解释，尤其在时间序列预测领域。虽然贝叶斯优化（BO）是一种标准方法，但它通常将调优任务视为独立处理，且对其决策提供的解释有限。大型语言模型（LLM）的最新进展为将结构化先验知识与推理融入优化流程提供了新的机遇。我们提出了LLM-AutoOpt，一种结合BO与基于LLM的上下文推理的混合HPO框架。该框架将数据集元特征、模型描述、历史优化结果及目标函数编码为LLM提示中的结构化元知识，并利用BO初始化搜索以缓解冷启动效应。这一设计实现了上下文感知且稳定的超参数精调，同时揭示了优化决策背后的推理过程。在多变量时间序列预测基准上的实验表明，相较于无元知识的BO和LLM基线方法，LLM-AutoOpt取得了更优的预测性能和更具可解释性的优化行为。