We present a low-latency real-time audio game commentary system that generates spoken commentary directly from live gameplay video. In this end-to-end setting, a key bottleneck is accumulated waiting time; conventional pipelines capture frames, generate text, and synthesize speech sequentially for each utterance, and do not request the next generation until speech playback has completed. This strict sequentiality causes long and unnatural silence between utterances. To address this latency bottleneck, our system runs text generation in parallel with speech playback and buffers multiple candidate utterances ahead of time, enabling immediate synthesis at playback boundaries. Experiments on fast-paced game videos show that our parallel design reduces the mean inter-utterance silence from 9.6 seconds to 0.3 seconds compared to sequential baselines. It also improves similarity to professional speaking--silence timing patterns by over 40 %, and a user study with 120 experienced game players confirms significantly improved perceived speaking rhythm. Our demo video is available at: https://youtu.be/pmrRUlvav8M.
翻译:我们提出一种低延迟的实时音频游戏解说系统,能够直接从实况游戏视频生成语音解说。在该端到端设置中,关键瓶颈在于累积等待时间:传统流水线按顺序对每个语句依次执行帧捕捉、文本生成与语音合成,且在语音播放完成前不会请求下一段生成。这种严格顺序性导致语句间出现漫长且不自然的静默。为突破此延迟瓶颈,本系统将文本生成与语音播放并行运行,并预缓存多个候选语句,从而在播放边界实现即时合成。针对快节奏游戏视频的实验表明,与顺序基线相比,我们的并行设计将语句间平均静默时间从9.6秒降至0.3秒;同时,与专业解说静默时序模式的相似度提升超过40%。一项包含120名资深游戏玩家的用户研究证实,感知到的解说节奏显著改善。演示视频详见:https://youtu.be/pmrRUlvav8M。