Tensor network (TN) is a powerful framework in machine learning, but selecting a good TN model, known as TN structure search (TN-SS), is a challenging and computationally intensive task. The recent approach TNLS~\cite{li2022permutation} showed promising results for this task, however, its computational efficiency is still unaffordable, requiring too many evaluations of the objective function. We propose TnALE, a new algorithm that updates each structure-related variable alternately by local enumeration, \emph{greatly} reducing the number of evaluations compared to TNLS. We theoretically investigate the descent steps for TNLS and TnALE, proving that both algorithms can achieve linear convergence up to a constant if a sufficient reduction of the objective is \emph{reached} in each neighborhood. We also compare the evaluation efficiency of TNLS and TnALE, revealing that $\Omega(2^N)$ evaluations are typically required in TNLS for \emph{reaching} the objective reduction in the neighborhood, while ideally $O(N^2R)$ evaluations are sufficient in TnALE, where $N$ denotes the tensor order and $R$ reflects the \emph{``low-rankness''} of the neighborhood. Experimental results verify that TnALE can find practically good TN-ranks and permutations with vastly fewer evaluations than the state-of-the-art algorithms.

翻译：张量网络是机器学习中一个强大的框架，但选择良好的张量网络模型（即张量网络结构搜索）是一项挑战性且计算密集的任务。近期方法TNLS在该任务中展现出良好前景，但其计算效率仍令人难以承受，需要过多目标函数评估。我们提出TnALE——一种通过局部枚举交替更新每个结构相关变量的新算法，与TNLS相比大幅减少了评估次数。我们从理论上研究TNLS与TnALE的下降步骤，证明若在每个邻域内目标函数达到充分下降，两种算法均可实现直至常数的线性收敛。我们进一步比较了TNLS与TnALE的评估效率，揭示为在邻域内达到目标函数下降，TNLS通常需要Ω(2^N)次评估，而TnALE理想情况下仅需O(N^2R)次评估，其中N表示张量阶数，R反映邻域的"低秩性"。实验结果表明，TnALE能以远少于现有最优算法的评估次数，找到实际高效的张量网络秩与排列。