Convolutional neural networks are now seeing widespread use in a variety of fields, including image classification, facial and object recognition, medical imaging analysis, and many more. In addition, there are applications such as physics-informed simulators in which accurate forecasts in real time with a minimal lag are required. The present neural network designs include millions of parameters, which makes it difficult to install such complex models on devices that have limited memory. Compression techniques might be able to resolve these issues by decreasing the size of CNN models that are created by reducing the number of parameters that contribute to the complexity of the models. We propose a compressed tensor format of convolutional layer, a priori, before the training of the neural network. 3-way kernels or 2-way kernels in convolutional layers are replaced by one-way fiters. The overfitting phenomena will be reduced also. The time needed to make predictions or time required for training using the original Convolutional Neural Networks model would be cut significantly if there were fewer parameters to deal with. In this paper we present a method of a priori compressing convolutional neural networks for finite element (FE) predictions of physical data. Afterwards we validate our a priori compressed models on physical data from a FE model solving a 2D wave equation. We show that the proposed convolutinal compression technique achieves equivalent performance as classical convolutional layers with fewer trainable parameters and lower memory footprint.

翻译：卷积神经网络现已广泛应用于图像分类、面部与物体识别、医学影像分析等多个领域。此外，在基于物理信息的模拟器等应用中，需要以最小延迟实现实时精确预测。当前神经网络设计包含数百万个参数，这使得在内存受限设备上部署此类复杂模型面临挑战。压缩技术可通过减少导致模型复杂性的参数数量来缩小CNN模型尺寸，从而有望解决这些问题。本文提出一种在神经网络训练之前对卷积层进行先验压缩的张量格式：将卷积层中的三阶核或二阶核替换为一阶滤波器。该方法还可减少过拟合现象。由于处理参数数量减少，使用原始卷积神经网络模型进行预测或训练所需的时间将显著降低。本文提出一种针对物理数据有限元预测的卷积神经网络先验压缩方法。随后，我们基于一个求解二维波动方程的有限元模型生成的物理数据对所提出的先验压缩模型进行验证。结果表明，所提卷积压缩技术能够在保持与经典卷积层相当性能的同时，减少可训练参数数量并降低内存占用。