Research On Information-Balanced Sampling And Optimization Reconstruction Methods In Video Compressed Sensing

Compressed sensing can complete compression for image and video signals during sampling,making it suitable for application scenarios with limited resources at the encoder.Restoring the original signal from the measurements has always been a major challenge in the compressed sensing field.Traditional iterative optimization-based algorithms can’t accomplish reconstruction in real time and have poor reconstruction quality at low sampling rates.Although the deep image and video compressed sensing methods have achieved a breakthrough in realtime reconstruction,there are still some problems that need to be solved,including the imbalance of sampling point information,the low efficiency of utilizing measurements and reference frame information,and the inability to achieve arbitrary sampling rate compressed sensing with high-quality reconstruction.To address these problems,this paper conducts research from image to video and from the single sampling rate model to the arbitrary sampling rate model.The details are as follows:1)A hybrid BCS sampling method(HBCS)based on information-balanced sampling is proposed to realize block-based linear full-image sampling.A gradient attention block(GABlock)is proposed to match HBCS and realize the adaptive fusion of measurement error gradient maps corresponding to block permutation sampling(BPS)and BCS.Finally,the hybrid sampling gradient attention network(HSGANet)is constructed to achieve high-quality reconstruction.The experiments fully demonstrate the effectiveness of the proposed algorithm.2)Combining deep learning with traditional iterative optimization algorithms,an unfolding motion compensation and residual reconstruction network(Umr Net)is proposed.The proposed motion compensation network(MCNet)can realize adaptive fusion among frames,correct the alignment of pixels such as deformation and occlusion,and improve the accuracy of motion compensation.The experiments show the superiority,strong real-time performance,and interpretability of Umr Net in reconstruction performance.3)The reference frames are introduced as conditional variables in the video frame prior.An optimization equation suitable for video compressed sensing reconstruction is derived using a Bayesian formula,and an iterative solution framework is derived based on the proximal gradient algorithm.According to this framework,the first optimization-inspired based unfolding proximal gradient motion compensation network(UPGM-Net)is designed to achieve multiple utilizations of measurements and reference frames through alternating gradient updates and inter-frame alignment.The experiments show that UPGM-Net can significantly improve the reconstruction performance of video compressed sensing.4)Based on the iterative solution framework for non-key frame optimization derived in the previous chapter,combining the multi-hypothesis idea with the optical flow estimation network,a multi-hypothesis aggregation module(MHA)is proposed.Multi-hypothesis sets are constructed by multiple groups of pixel alignment to improve the accuracy of inter-frame correlation modeling.In addition,the idea of optical flow optimization from coarse to fine estimation is introduced,and a deep unfolding multi-hypothesis aggregation network(DUMHAN)including dual iterative branches of frame reconstruction and optical flow estimation is proposed.Experiments show that DUMHAN can further improve reconstruction quality on the basis of UPGM-Net.5)For the first time,multiple priors(the measurements and the reference frame sets)are simultaneously used as conditional variables for the maximum posterior probability of non-key frames,and the optimization equation is derived using the Bayesian formula;the solution framework is derived by the proximal gradient algorithm,based on which the deep multiinformation flow gradient update and aggregation network(DMIGAN)is designed.DMIGAN,by jointly optimizing the observation error and alignment error,achieves the interactive utilization of observations and reference frames.Experiments demonstrate the superiority of DMIGAN in reconstruction performance.6)In order to achieve high-quality reconstruction at arbitrary sampling rates by a single model,an image/video compressed sensing reconstruction network training strategy of training image balancing and gradient balancing(TBGB)is proposed to balance the reconstruction performance at different sampling rates.Experiments show that the reconstruction performance of an arbitrary sampling rate image/video compressed sensing model trained using TBGB is comparable to that of a model trained independently.For the design of the reconstruction network,the design criteria of inter-level structure difference and intra-level loop iteration are further proposed to reduce the number of model parameters and improve the ability to recover high-frequency information.