Interactions among large number of entities is naturally high-dimensional and incomplete (HDI) in many big data related tasks. Behavioral characteristics of users are hidden in these interactions, hence, effective representation of the HDI data is a fundamental task for understanding user behaviors. Latent factor analysis (LFA) model has proven to be effective in representing HDI data. The performance of an LFA model relies heavily on its training process, which is a non-convex optimization. It has been proven that incorporating local curvature and preprocessing gradients during its training process can lead to superior performance compared to LFA models built with first-order family methods. However, with the escalation of data volume, the feasibility of second-order algorithms encounters challenges. To address this pivotal issue, this paper proposes a mini-block diagonal hessian-free (Mini-Hes) optimization for building an LFA model. It leverages the dominant diagonal blocks in the generalized Gauss-Newton matrix based on the analysis of the Hessian matrix of LFA model and serves as an intermediary strategy bridging the gap between first-order and second-order optimization methods. Experiment results indicate that, with Mini-Hes, the LFA model outperforms several state-of-the-art models in addressing missing data estimation task on multiple real HDI datasets from recommender system. (The source code of Mini-Hes is available at https://github.com/Goallow/Mini-Hes)

翻译：在大数据相关任务中，大量实体间的交互天然具有高维且不完全（HDI）的特性。用户行为特征隐藏在这些交互之中，因此对HDI数据的有效表示是理解用户行为的基础任务。潜因子分析（LFA）模型已被证明能有效表述HDI数据。LFA模型的性能高度依赖其训练过程，而该过程属于非凸优化问题。已有研究证明，在训练过程中引入局部曲率信息并对梯度进行预处理，可构建出性能优于基于一阶方法的LFA模型。然而，随着数据规模的增长，二阶算法的可行性面临挑战。针对这一关键问题，本文提出一种用于构建LFA模型的微型块对角无海森（Mini-Hes）优化方法。该方法基于对LFA模型海森矩阵的分析，利用广义高斯-牛顿矩阵中的主对角块，作为衔接一阶与二阶优化方法的中间策略。实验结果表明，在推荐系统多个真实HDI数据集上，采用Mini-Hes的LFA模型在缺失数据估计任务中优于多个当前最优模型。（Mini-Hes源代码见https://github.com/Goallow/Mini-Hes）