In video game design, audio (both environmental background music and object sound effects) play a critical role. Sounds are typically pre-created assets designed for specific locations or objects in a game. However, user-generated content is becoming increasingly popular in modern games (e.g. building custom environments or crafting unique objects). Since the possibilities are virtually limitless, it is impossible for game creators to pre-create audio for user-generated content. We explore the use of generative artificial intelligence to create music and sound effects on-the-fly based on user-generated content. We investigate two avenues for audio generation: 1) text-to-audio: using a text description of user-generated content as input to the audio generator, and 2) image-to-audio: using a rendering of the created environment or object as input to an image-to-text generator, then piping the resulting text description into the audio generator. In this paper we discuss ethical implications of using generative artificial intelligence for user-generated content and highlight two prototype games where audio is generated for user-created environments and objects.

翻译：在视频游戏设计中，音频（包括环境背景音乐和物体音效）起着关键作用。声音通常是针对游戏中的特定位置或物体预先创建的资源。然而，用户生成内容在现代游戏中越来越受欢迎（例如，构建自定义环境或制作独特物体）。由于可能性几乎无穷无尽，游戏创作者无法为用户生成内容预先创建音频。我们探索利用生成式人工智能根据用户生成内容即时创建音乐和音效。我们研究了两种音频生成途径：1）文本到音频：将用户生成内容的文本描述作为音频生成器的输入；2）图像到音频：将所创建环境或物体的渲染图作为图像到文本生成器的输入，然后将生成的文本描述导入音频生成器。本文讨论了将生成式人工智能用于用户生成内容的伦理影响，并重点介绍了两款原型游戏，在这些游戏中，音频是为用户创建的环境和物体生成的。