Study On The Problems Of Image And Video Compressive Sensing

With the arrival of the big data era and the requirements of people for the quality of multimedia data,such as images and videos,the production of various data is also increasing explosively.As a result,it makes increasingly difficult to store and transmit data.Compressive sensing is a way to solve this problem.It directly stores the compressed aliased signal with its unique aliasing sampling method,transmits the aliased signal directly to the receiver,and accurately recovers the scene signal through the reconstruction algorithm.This manner effectively relieves the pressure of storage and transmission.Due to the effectiveness of compressive sensing,it has been applied in various kinds of signal processing fields such as medical imaging,high-speed photography,remote sensing detection and other fields.There are two kinds of existing compressive sensing algorithms: model-driven algorithm and data-driven algorithm.The model-driven algorithm mainly designs the measurement matrix and the corresponding parametric models through the prior knowledge of the scene,and obtains the reconstruction signal through the optimization algorithm.Most of the datadriven algorithms are based on the deep learning network,which need training data to train the measurement matrix and the corresponding reconstruction algorithm.The model-driven method is difficult to be applied in actual tasks due to its high time complexity.Compared with the model-driven method,the data-driven algorithm can achieve better reconstruction effect under the low time complexity.In this dissertation,we focus on the existing problems of data-driven algorithm.Firstly,although the existing CNN-based CS methods achieve encouraging performance,there is no clear understanding of how they perform so well due to the "black box" characteristics of the deep network model.This is because there is no visualization of CNNbased compressive sensing framework so far.Therefore,it is urgent to study the principle of the algorithm and visualize it concretely.Secondly,most of the compressive sensing algorithms do not distinguish the primary and secondary regions in the scene when they measure the static scene.However,when people observe the scene,we will instinctively draw out the regions of interest and the regions of non-interest according to our own degree of interest.Then the regions of interest will be paid more attention to and the regions of noninterest will be paid less attention to.Therefore,the algorithms that imitate human vision system need to be developed.Finally,for the measurement manner and reconstruction algorithm of dynamic scene,the existing methods generally only measure the scene on the time or space dimension,which leads to the large redundancy of the non-compression dimension.Therefore,the compressive sensing sampling and recovery algorithm for spatiotemporal dimension need to be proposed.According to the problems above,we carry out researches and propose corresponding solutions.The main research contents and contributions of this dissertation include the following three aspects:1.A compressive sensing network based on the residual block is proposed.We take the proposed framework as an example of visualization to further analyze the working mechanism of compressive sensing network.This is the first time to have an insight look into the CS network.Specifically,the measurement matrix is visualized and analyzed in time and frequency domains.Then,the process of network image reconstruction is analyzed from the perspective of space and system.Then,through exploratory experiments,we further explore the role of deep residual module in the network.Finally,compared with the existing methods,the superiority of the proposed algorithm is proved.2.A compressive sensing network based on ROI(Region of Interest),ROI-CSNet,is proposed.The proposed network can achieve a higher reconstruction quality in the ROIs while retaining the scientific quality in the rest of the image,under the constraints of measurements budget.This is the first work of ROI-aware image reconstruction using the deep convolution neural network.Compared with the traditional compressive sensing method of uniform sampling,this method can obtain higher quality ROI-aware image in real time.Secondly,for the sensing procedure,our method consists of preliminary and ROI sensing procedures.In the preliminary sensing procedure,the full scene is measured for extraction of ROIs.After that,more sensing resources are allocated to ROIs of the scene in the ROI sensing procedure.For the ROI recovery,we put the measurements of these two sensing procedures together to recover the ROI-aware image.In addition,the measurement and reconstruction processes are jointly trained,so that the two measurement matrices can capture the complementary information in the scene.Finally,in order to improve the quality of ROI-aware image recovery,we design the ROI-aware loss function,which makes the network pay more attention to ROI region during training and testing.3.A hybrid-3D video compressive sensing network based on spatio-temporal measurement and reconstruction is proposed.The framework consists of measurement part and reconstruction part.In the measurement part,the learned spatio-temporal measurement matrix is described mathematically.In the reconstruction part,a reconstruction sub-network composed of hybid-3D residual block is designed.This structure enables the network to intuitively represent the spatio-temporal feature,and significantly reduces the network parameters.At the same time,through a series of experiments,we explore the kernel size and block number of the hybrid-3D residual block.Finally,compared with other methods,the proposed method achieves a better reconstruction performance and higher speed on the public dataset.