Mixture-of-Experts (MoE) architectures leverage sparse activation to enhance the scalability of large language models (LLMs), making them suitable for deployment in resource-constrained edge networks. However, the sheer number of experts often exceeds the memory capacity of individual edge nodes, necessitating wireless distributed MoE (WIDE) inference where experts are spread across multiple edge nodes. In this context, expert selection directly affects communication costs. Motivated by the similarity of experts, we propose SiftMoE, which judiciously selects or skips experts to strike a tradeoff between communication costs and inference accuracy. Specifically, we first establish theoretical bounds on the accuracy degradation resulting from expert replacement or skipping. Based on the bounds, we formulate an energy minimization problem for expert selection in WIDE inference subject to latency and accuracy constraints. In particular, for slow-fading channels, we derive optimal expert selection policies for both single-token decoding and multi-token prefilling. For fast-fading channels, we further extend our scheme to cope with rapidly varying channel conditions. Simulation results demonstrate that SiftMoE significantly reduces energy consumption while maintaining inference accuracy compared with conventional Top-K routing in WIDE systems.

翻译：混合专家（MoE）架构利用稀疏激活机制提升了大语言模型（LLM）的可扩展性，使其适用于资源受限的边缘网络部署。然而，专家数量的庞大往往超过单个边缘节点的内存容量，催生了将专家分布到多个边缘节点的无线分布式MoE（WIDE）推理范式。在此背景下，专家选择直接影响了通信成本。受专家相似性启发，我们提出SiftMoE，通过审慎选择或跳过专家以平衡通信成本与推理精度。具体而言，我们首先建立了因专家替换或跳过导致的精度退化理论边界。基于该边界，我们构建了满足延迟与精度约束的WIDE推理中专家选择的能耗最小化问题。特别地，针对慢衰落信道，我们分别推导了单令牌解码和多令牌预填充情况下的最优专家选择策略。针对快衰落信道，我们进一步扩展方案以应对快速变化的信道条件。仿真结果表明，与WIDE系统中的传统Top-K路由相比，SiftMoE在保持推理精度的同时显著降低了能耗。