Automatic prompt optimization (APO) has emerged as a powerful paradigm for improving LLM performance without manual prompt engineering. Reflective APO methods such as GEPA iteratively refine prompts by diagnosing failure cases, but the optimization process remains black-box and label-free, leading to uninterpretable trajectories and systematic failure. We identify and empirically demonstrate four limitations: on GSM8K with a defective seed, GEPA degrades accuracy from 23.81% to 13.50%. We propose VISTA, a multi-agent APO framework that decouples hypothesis generation from prompt rewriting, enabling semantically labeled hypotheses, parallel minibatch verification, and interpretable optimization trace. A two-layer explore-exploit mechanism combining random restart and epsilon-greedy sampling further escapes local optima. VISTA recovers accuracy to 87.57% on the same defective seed and consistently outperforms baselines across all conditions on GSM8K and AIME2025.

翻译：自动提示优化（APO）已成为一种无需手动提示工程即可提升大语言模型性能的强大范式。诸如GEPA之类的反思式APO方法通过诊断失败案例来迭代优化提示，但其优化过程仍保持黑箱且无标签，导致轨迹不可解释和系统性失败。我们识别并实证了四种局限性：在GSM8K数据集上使用有缺陷的种子时，GEPA的准确率从23.81%降至13.50%。我们提出VISTA，一种多智能体APO框架，将假设生成与提示重写解耦，实现了语义化标签假设、并行小批量验证以及可解释的优化轨迹。结合随机重启与epsilon-贪婪采样的双层探索-利用机制进一步摆脱了局部最优。VISTA在同样的有缺陷种子上将准确率恢复至87.57%，并在GSM8K和AIME2025数据集的所有条件下持续优于基线方法。