This paper introduces CLIPSwarm, a new algorithm designed to automate the modeling of swarm drone formations based on natural language. The algorithm begins by enriching a provided word, to compose a text prompt that serves as input to an iterative approach to find the formation that best matches the provided word. The algorithm iteratively refines formations of robots to align with the textual description, employing different steps for "exploration" and "exploitation". Our framework is currently evaluated on simple formation targets, limited to contour shapes. A formation is visually represented through alpha-shape contours and the most representative color is automatically found for the input word. To measure the similarity between the description and the visual representation of the formation, we use CLIP [1], encoding text and images into vectors and assessing their similarity. Subsequently, the algorithm rearranges the formation to visually represent the word more effectively, within the given constraints of available drones. Control actions are then assigned to the drones, ensuring robotic behavior and collision-free movement. Experimental results demonstrate the system's efficacy in accurately modeling robot formations from natural language descriptions. The algorithm's versatility is showcased through the execution of drone shows in photorealistic simulation with varying shapes. We refer the reader to the supplementary video for a visual reference of the results.

翻译：本文提出CLIPSwarm，一种基于自然语言自动建模无人机群编队的新算法。该算法首先对输入单词进行语义丰富，构建文本提示作为迭代方法的输入，以寻找与给定单词最匹配的编队。算法通过"探索"与"利用"两个不同阶段，迭代优化机器人编队使其与文本描述对齐。当前框架仅针对简单轮廓形状的编队目标进行验证。编队通过alpha-shape轮廓进行可视化呈现，并自动提取输入单词最具代表性的颜色。为评估文本描述与编队视觉表征之间的相似度，我们采用CLIP[1]模型将文本与图像编码为向量并计算相似性。随后，算法在可用无人机数量约束下重新排列编队，以更有效地视觉呈现该单词。最后为无人机分配控制指令，确保机器人行为与无碰撞运动。实验结果表明，该系统能够准确基于自然语言描述建模机器人编队。通过在具有不同形状的光照仿真中执行无人机表演，验证了算法的多功能性。读者可参阅补充视频获取结果的可视化参考。