Constrained Fusion Method Of Multi-Source Data Based On Deep Learning

In the process of industrial production,a large number and variety of sensors are often used to monitor the operating status of the system,and methods such as data fusion are used to mine the potential information features hidden behind the data,which can provide useful guidance for system analysis and optimal decision.However,due to factors such as diverse sensor types and complex and changeable system operating conditions,most of the monitored multi-source data exhibits properties such as spatiotemporal disparity and heterogeneity in form,which brings difficulties to multi-source data fusion and decision.Therefore,this research focuses on the multi-modal,unbalanced,and heterogeneous structural features of multi-source data,and studies multi-source data fusion methods based on deep constrained representation learning.The specific research contents include:(1)Aiming at the problem that key information is lost due to the lack of timely mining of interactive information between modalities during multimodal data fusion,a multimodal data fusion method based on constrained adversarial convolutional auto-encoding memory fusion network is studied.In this method,a squeeze excitation module is firstly introduced into the convolutional autoencoder,so that it can focus more on fault information while extracting independent features.Secondly,a modal discriminator and a similarity metric constraint function are introduced to facilitate the encoder to capture the interaction information and modal invariance features between modalities,thereby reducing the difference between multimodal distributions.Then,the long-short-term memory network is used to fuse the independent features and interaction features to capture the contextual information features of multi-source data.Finally,the effectiveness of the proposed model is verified by comparative experiments,ablation studies,and generalization performance experiments.(2)Aiming at the problem that the generalization ability of the model is reduced due to the uneven sample categories in multimodal data fusion,an unbalanced data fusion method based on the invariant spatiotemporal attention fusion network is studied.The method first pre-trains the convolutional autoencoder with balanced data to reduce the influence of eccentric learning caused by unbalanced samples on the network.Second,a convolutional attention module,a long-short-term memory network,and a similarity constraint function are introduced into the pretrained convolutional autoencoder to capture the independent and interactive features with spatiotemporal features.Then,the triplet boundary constraint function and the multilayer perceptron are used to promote the clustering and fusion of each modal feature information.Finally,the effectiveness of the proposed model is verified by comparative experiments,ablation studies,and generalization performance experiments.(3)Aiming at the problems of insufficient feature representation and difficulty in determining the contribution of each modal to the task during heterogeneous data fusion,a heterogeneous data fusion method based on dynamic invariant-specific representation fusion network is studied.The method firstly improves the domain separation network by expanding the modes,adding constraints and separating feature associations,so as to capture the characteristics of each mode and effectively utilize the redundant information between modes.Secondly,the cosine triplet constraint is added to the improved domain separation network to improve the semantic clustering effect of each modality.Then,each representation is dynamically fused using a hierarchical graph fusion network to adaptively acquire modal interaction information.Finally,this thesis conducts comparative experiments and designs fusion strategies,loss function ablation,and similar loss function analysis experiments.The experimental results verify the effectiveness of proposed method and loss function.