Tensor network (TN) is a powerful framework in machine learning, but selecting a good TN model, known as TN structure search (TN-SS), is a challenging and computationally intensive task. The recent approach TNLS~\cite{li2022permutation} showed promising results for this task, however, its computational efficiency is still unaffordable, requiring too many evaluations of the objective function. We propose TnALE, a new algorithm that updates each structure-related variable alternately by local enumeration, \emph{greatly} reducing the number of evaluations compared to TNLS. We theoretically investigate the descent steps for TNLS and TnALE, proving that both algorithms can achieve linear convergence up to a constant if a sufficient reduction of the objective is \emph{reached} in each neighborhood. We also compare the evaluation efficiency of TNLS and TnALE, revealing that $\Omega(2^N)$ evaluations are typically required in TNLS for \emph{reaching} the objective reduction in the neighborhood, while ideally $O(N^2R)$ evaluations are sufficient in TnALE, where $N$ denotes the tensor order and $R$ reflects the \emph{``low-rankness''} of the neighborhood. Experimental results verify that TnALE can find practically good TN-ranks and permutations with vastly fewer evaluations than the state-of-the-art algorithms.

翻译：张量网络（TN）是机器学习中的强大框架，但选择好的TN模型（称为TN结构搜索（TN-SS））是一项具有挑战性且计算量大的任务。近期方法TNLS~\cite{li2022permutation}在此任务上展示了有希望的结果，然而其计算效率仍然难以承受，需要对目标函数进行过多评估。我们提出TnALE，一种新算法，通过局部枚举交替更新每个结构相关变量，与TNLS相比显著减少了评估次数。我们从理论上研究了TNLS和TnALE的下降步骤，证明若在每个邻域内达到目标函数充分下降，两种算法均可实现线性收敛（至多常数误差）。我们还比较了TNLS和TnALE的评估效率，揭示在TNLS中为在邻域内实现目标函数下降通常需要$\Omega(2^N)$次评估，而在TnALE中理想情况下$O(N^2R)$次评估就足够，其中$N$表示张量阶数，$R$反映邻域的“低秩性”。实验结果表明，与最先进算法相比，TnALE能用少得多的评估次数找到实际有效的TN秩和置换。