Research On Multimodal Feature Fusion Algorithm Based On Canonical Correlation Analysis

In the context of big data and the Internet era,the ways of data acquisition are diverse,and multimodal data has become the main form of data existence in various industries.As a key technology for multimodal data learning,feature fusion has also received widespread attention in research fields such as pattern recognition and machine learning.As a representative method of multimodal feature fusion,canonical correlation analysis aims to learn the relevant projection directions of two sets of modal data and obtain more discriminative related fusion features in low dimensional subspaces.At present,canonical correlation analysis and its variants have been widely applied in fields such as image processing,information fusion,bioinformatics,etc.However,as a linear method for processing two modalities,it still faces the problem of being unable to reveal the nonlinear complex structure of multiple modal data and being unable to adapt to clustering methods.In response to the above issues,this article takes canonical correlation analysis as the foundation,with feature fusion as the direction,fuses local information,establishes a new optimization model for locally enhanced feature fusion,and further extends it to multimodal canonical correlation analysis,proposing a clustering adaptive feature fusion method.The specific research work is as follows:(1)In order to effectively capture local structural information in raw highdimensional data,this dissertation proposes an orthogonal feature fusion method based on local enhancement,namely cross modal local orthogonal canonical correlation analysis.This method integrates local neighborhood information and orthogonal constraints of the projection transformation matrix,establishes cross modal local enhanced dispersion,and constructs an optimization model for local enhanced orthogonal feature fusion.With the help of local enhancement structure,this method can effectively mine the local structural information hidden in the data when facing complex nonlinear problems,reveal the intrinsic manifold structure of the data,overcome the high dependence on the distribution of the original data,and better preserve the reconstruction relationships between the data.The obtained low dimensional correlated fusion features are more discriminative.A large number of experimental results on facial and multi feature datasets have verified the effectiveness of this method.(2)In order to solve the problem of difficulty in adapting multimodal feature fusion algorithms and clustering methods,this dissertation proposes a clustering adaptive multimodal canonical correlation analysis method.This method constructs a unified optimization model for multimodal canonical correlation analysis and clustering.The low dimensional correlated fusion features obtained in feature fusion methods can further improve clustering performance,while the category labels in clustering methods can further constrain the correlated projection direction of feature fusion.This method not only achieves discriminative learning of relevant projection directions under unsupervised conditions,but also directly obtains category labels of high-dimensional multimodal data.In addition,this method also achieves out of sample extension in class labels,optimizes the solution through iteration,and analyzes the convergence of iteration in theory and experiment.The effectiveness of this method has been proven by a large number of experimental results on multi feature,multi language,and image datasets.Figure [8] Table [12] Reference [74]...