Deep Learning Recommendation Models (DLRMs) underpin personalized services but face a critical freshness-accuracy tradeoff due to massive parameter synchronization overheads. Production DLRMs deploy decoupled training/inference clusters, where synchronizing petabyte-scale embedding tables (EMTs) causes multi-minute staleness, degrading recommendation quality and revenue. We observe that (1) inference nodes exhibit sustained CPU underutilization (peak <= 20%), and (2) EMT gradients possess intrinsic low-rank structure, enabling compact update representation. We present LiveUpdate, a system that eliminates inter-cluster synchronization by colocating Low-Rank Adaptation (LoRA) trainers within inference nodes. LiveUpdate addresses two core challenges: (1) dynamic rank adaptation via singular value monitoring to constrain memory overhead (<2% of EMTs), and (2) NUMA-aware resource scheduling with hardware-enforced QoS to eliminate update inference contention (P99 latency impact <20ms). Evaluations show LiveUpdate reduces update costs by 2x versus delta-update baselines while achieving higher accuracy within 1-hour windows. By transforming idle inference resources into freshness engines, LiveUpdate delivers online model updates while outperforming state-of-the-art delta-update methods by 0.04% to 0.24% in accuracy.

翻译：深度学习推荐模型（DLRMs）支撑着个性化服务，但由于海量参数同步开销，面临关键的新鲜度-准确性权衡。生产环境中的DLRMs采用解耦的训练/推理集群架构，同步PB级嵌入表（EMTs）会导致数分钟的陈旧性，从而降低推荐质量和收入。我们观察到：（1）推理节点持续存在CPU利用率不足（峰值≤20%）；（2）EMT梯度具有内在的低秩结构，可实现紧凑的更新表示。本文提出LiveUpdate系统，通过在推理节点内部署低秩自适应（LoRA）训练器，消除集群间同步开销。LiveUpdate解决了两个核心挑战：（1）通过奇异值监测实现动态秩自适应，以控制内存开销（<EMTs的2%）；（2）采用NUMA感知的资源调度与硬件强化的服务质量保障，消除更新对推理的干扰（P99延迟影响<20毫秒）。评估表明，LiveUpdate相比增量更新基线将更新成本降低2倍，并在1小时窗口内实现更高的准确性。通过将闲置推理资源转化为新鲜度引擎，LiveUpdate在提供在线模型更新的同时，其准确性较最先进的增量更新方法提升0.04%至0.24%。