Nonlinear Generalized Principal Component Analysis

With the continuous upgrading of communication technology and the continuous deepening of digital transformation,the total amount of data generated by the economy and society increase unceasingly,and the types of data are also increasingly rich.At the same time,high-order tensors appear more and more frequently in people’s vision.In the background,data compression has received extensive attention in the field of statistical research.After summarizing and analysing,it is found that the data compression methods commonly used are either limited by the linear assumption of the model,or cannot be expressed in formulas,or the model interpretability is weak.In order to overcome these problems,based on the linear principal component analysis method,this paper proposes a nonlinear data compression method that can be display in expressions and can be interpret—Nonlinear Generalized Principal Component Analysis(NGPCA).Due to the more complex spatial structure of high-order tensors,the linear algebraic operations of low-order tensors are no longer applicable.Therefore,this paper takes both low-order and high-order situations into consideration,and respectively introduces Low-Order Nonlinear Generalized Principal Component Analysis method(LO-NGPCA)and High-Order Nonlinear Generalized Principal Component Analysis method(HO-NGPCA).Specifically,it includes the following two parts:First,for low-order tensors,this paper designs the LO-NGPCA method,and takes second-order data as an example to illustrate the method.Based on the principal component analysis,the activation function is introduced to map the projected data,and the interpretation is obtained from the perspective of network model.By introducing deformation sub-layer at a specific position,the direction to compress is changed.In addition,Low-Order Deformable Back Propagation algorithm(LO-DBP)is designed to estimate the parameters of the model.Finally,the numerical experiments based on the ORL database show that the algorithm has convergence and the compression performance of LONGPCA is better than that of linear principal component analysis methods,including principal component analysis,two-dimensional principal component analysis and generalized principal component analysis under the same or more severe compression conditions.Second,for high-order tensors,considering that the method designed in this paper will not undergo essential changes with the increase of the order,so taking the third-order tensor as an example to describe the HO-NGPCA method.The relevant basic knowledge is first introduced in this chapter and then according to the difference of the selected set to compress,the HO-NGPCA method with a depth of 1 and a depth of is described respectively.Not only the internal relation of two HO-NGPCA methods with different depth is clarified,but also the corresponding network model is constructed to explain the method intuitively.The parameter estimation is based on the High-Order Deformable Back Propagation algorithm(HO-DBP).Finally,through numerical simulations,the convergence of the HO-DBP algorithm and the superiority of the HO-NGPCA method in compression performance are respectively illustrated.