Tensor network (TN) is a powerful framework in machine learning, but selecting a good TN model, known as TN structure search (TN-SS), is a challenging and computationally intensive task. The recent approach TNLS~\cite{li2022permutation} showed promising results for this task, however, its computational efficiency is still unaffordable, requiring too many evaluations of the objective function. We propose TnALE, a new algorithm that updates each structure-related variable alternately by local enumeration, \emph{greatly} reducing the number of evaluations compared to TNLS. We theoretically investigate the descent steps for TNLS and TnALE, proving that both algorithms can achieve linear convergence up to a constant if a sufficient reduction of the objective is \emph{reached} in each neighborhood. We also compare the evaluation efficiency of TNLS and TnALE, revealing that $\Omega(2^N)$ evaluations are typically required in TNLS for \emph{reaching} the objective reduction in the neighborhood, while ideally $O(N^2R)$ evaluations are sufficient in TnALE, where $N$ denotes the tensor order and $R$ reflects the \emph{``low-rankness''} of the neighborhood. Experimental results verify that TnALE can find practically good TN-ranks and permutations with vastly fewer evaluations than the state-of-the-art algorithms.

翻译：张量网络(TN)是机器学习中的强大框架，但选择良好的张量网络模型（即张量网络结构搜索TN-SS）是一项具有挑战性且计算密集的任务。近期方法TNLS~\cite{li2022permutation}在此任务中展现了有前景的结果，但其计算效率仍难以承受，需要对目标函数进行过多评估。我们提出TnALE算法，通过局部枚举交替更新每个结构相关变量，相较于TNLS大幅减少了评估次数。我们从理论上研究了TNLS和TnALE的下降步骤，证明若在每个邻域内实现目标函数的充分下降，两种算法均可达到线性收敛至常数。我们还比较了TNLS与TnALE的评估效率，揭示TNLS在邻域内实现目标下降通常需要$\Omega(2^N)$次评估，而TnALE理论上仅需$O(N^2R)$次评估即可满足要求，其中$N$表示张量阶数，$R$反映邻域的"低秩性"。实验结果表明，与现有最优算法相比，TnALE能以极少的评估次数找到实际有效的张量网络阶数排列。