Research On Key Technologies Of Multimodal Data Fusion And Transmission

Cyber-Physical System,also known as CPS,is a complex system consisting of perception,computation,transmission and control.As an important part of CPS,wireless sensor network(WSN)is responsible for observing and perceiving the real environment,and transmitting the perception data to the upper application of CPS through the network,so that the CPS can understand the current state of the observed environment,and make reasonable decisions and responses based on real-time observation data.The data collected by WSN often involves multiple modals,and there are large amounts of data and redundant information.Considering the energy limitation of WSN,in order to achieve efficient transmission of multimodal sensing data,ensure quality of service and reduce network energy consumption,this thesis adopts the method of fusing multimodal sensing data first,then transmitting fused information to reduce the overall energy consumption of the network,and promote the extension of network life.The main work of this thesis is as follows:(1)A multimodal data fusion algorithm is proposed.In order to solve the common problem of missing values in WSN data acquisition,the algorithm first uses the improved stacked autoencoder to fill in missing values to obtain complete data,so as to avoid the loss of important information caused by discarding the collected data with missing values.Then,an improved deep belief network and an autoencoder are used to learn the fused representation of interpolated multimodal data to obtain fused information.(2)Design an opportunistic routing algorithm considering delay,energy consumption and transmission reliability to transmit multimodal fused information.The routing algorithm defines QoS metrics considering energy efficiency and transmission delay,and calculates the corresponding priority for each neighbor node.Then,referring to the user's expected delay and reliability requirements,some neighbor nodes are selected as candidate nodes for forwarding data.Finally,the forwarding node determination and cooperation strategies are proposed,and an optimal node is selected from the candidate nodes to successfully forward the data to the next hop.(3)Design and implement the simulation platform of fusion-transmission algorithm.Combining the above two algorithms to form a fusion-transmission algorithm,and taking the fusion and transmission of multimodal environmental monitoring data as an example,the workflow and simulation results of the algorithm are presented.