Large Language Models (LLMs) like GPT-3 have sparked significant interest in their generative capabilities, leading to the development of various commercial applications. The high cost of using the models drives application builders to maximize the value of generation under a limited inference budget. This paper presents a study of optimizing inference hyperparameters like the number of responses, temperature and max tokens, which significantly affects the utility/cost of text generation. We design a framework named EcoOptiGen which leverages economical hyperparameter optimization and cost-based pruning. Experiments with the latest GPT-3.5 models on a variety of tasks verify its effectiveness. EcoOptiGen is implemented in the FLAML library: https://github.com/microsoft/FLAML, and we provide one example of using it at: https://microsoft.github.io/FLAML/docs/Examples/Integrate%20-%20OpenAI.

翻译：大型语言模型（LLMs）如GPT-3引发了对其生成能力的极大关注，进而推动了多种商业应用的开发。由于使用这些模型的成本高昂，应用开发者必须在有限的推理预算内最大化生成的价值。本文研究了优化推理超参数（如响应数量、温度参数和最大令牌数）的方法，这些参数对文本生成的效用/成本有显著影响。我们设计了一个名为EcoOptiGen的框架，该框架利用经济高效的超参数优化和基于成本剪枝策略。在多种任务上使用最新的GPT-3.5模型进行的实验验证了其有效性。EcoOptiGen已在FLAML库中实现（https://github.com/microsoft/FLAML），我们提供了一个使用示例：https://microsoft.github.io/FLAML/docs/Examples/Integrate%20-%20OpenAI。