Beyond diagonal reconfigurable intelligent surfaces (BD-RIS) can realize the interconnection between reflecting elements through the impedance network, thereby providing a new approach for the performance improvement of integrated sensing and communication (ISAC) systems. This paper investigates the optimization problem of BD-RIS-aided multiuser ISAC system, aiming to achieve the flexible design of trade-offs between communication and sensing performance. Specifically, we propose an optimization framework jointly combining the multiuser interference management and sensing beam gain approximation method. By jointly optimizing the precoding vector and RIS phase-shift matrix, improving the multiuser communication sum rate through the proposed interference management method, and enhancing the system sensing performance through the beam gain approximation method. For the resulting non-convex weighted optimization problem, we employ the alternating optimization (AO) algorithm to decouple it into two subproblems of precoding vector and phase-shift matrix optimization, with each step admitting closed-form solutions.Simulation results demonstrate that the proposed BD-RIS-aided ISAC system can achieve significant improvement in the trade-offs between communication and sensing performance than the traditional diagonal RIS, verifying the effectiveness of the proposed optimization framework.

翻译：超对角可重构智能表面（BD-RIS）通过阻抗网络实现反射单元间的互连，从而为提升集成感知与通信（ISAC）系统的性能提供了新途径。本文研究了BD-RIS辅助的多用户ISAC系统的优化问题，旨在实现通信与感知性能之间权衡的灵活设计。具体而言，我们提出了一个联合多用户干扰管理与感知波束增益逼近方法的优化框架。通过联合优化预编码向量与RIS相移矩阵，利用所提干扰管理方法提升多用户通信和速率，并通过波束增益逼近方法增强系统感知性能。针对由此产生的非凸加权优化问题，我们采用交替优化（AO）算法将其解耦为预编码向量与相移矩阵优化两个子问题，其中每一步均可获得闭式解。仿真结果表明，与传统对角RIS相比，所提出的BD-RIS辅助ISAC系统能够在通信与感知性能的权衡上取得显著提升，验证了所提优化框架的有效性。