Ragnarock is a virtual reality (VR) rhythm game in which you play a Viking captain competing in a longship race. With two hammers, the task is to crush the incoming runes in sync with epic Viking music. The runes are defined by a beat map which the player can manually create. The creation of beat maps takes hours. This work aims to automate the process of beat map creation, also known as the task of learning to choreograph. The assignment is broken down into three parts: determining the timing of the beats (action placement), determining where in space the runes connected with the chosen beats should be placed (action selection) and web-application creation. For the first task of action placement, extraction of predominant local pulse (PLP) information from music recordings is used. This approach allows to learn where and how many beats are supposed to be placed. For the second task of action selection, Recurrent Neural Networks (RNN) are used, specifically Gated recurrent unit (GRU) to learn sequences of beats, and their patterns to be able to recreate those rules and receive completely new levels. Then the last task was to build a solution for non-technical players, the task was to combine the results of the first and the second parts into a web application for easy use. For this task the frontend was built using JavaScript and React and the backend - python and FastAPI.
翻译:Ragnarock是一款虚拟现实(VR)节奏游戏,玩家扮演维京船长参与长船竞速赛。玩家需手持两柄战锤,在史诗级维京音乐伴奏下同步击碎来袭符文。这些符文由玩家可手动创建的节拍图谱定义,但制作节拍图谱需要耗费数小时。本研究旨在自动化节拍图谱创建流程,即习得编排动作的任务。该任务被分解为三个子问题:确定节拍时间节点(动作定位)、确定所选节拍对应符文的空间位置(动作选择)以及网页应用开发。针对第一个动作定位任务,采用音乐录音中主位脉冲信息提取方法。该方案能够学习节拍应放置的位置及数量。针对第二个动作选择任务,采用循环神经网络,具体使用门控循环单元学习节拍序列及其模式规律,以复现这些规则并生成全新关卡。最后一项任务是为非技术背景玩家构建解决方案:将前两部分结果整合为易用的网页应用。该前端采用JavaScript与React构建,后端使用Python与FastAPI实现。