Inter-reflection Compensation Of Immersion Projection Images Based On Convolution Neural Network

The immersive projection system is widely applied in the fields of human-computer interaction,augmented reality and virtual reality.The surround projection screen and multiprojector technology are usually used to achieve stereoscopic and real vision.Due to the nonplanar nature of the projection screen and the large number of projectors,the projection light generates complex inter-reflection during the propagation,resulting in luminance redundancy and quality degradation of the projection images.It will disturb the user's immersive experience,and even seriously hinder the application of virtual reality and other systems.So inter-reflection compensation has important practical value to improve the quality of the projection images.The traditional methods of inter-reflection compensation are usually multi-stage,which lead to a low accuracy and poor compensation effect.Convolutional neural networks use an end-to-end solution to solve image enhancement tasks effectively.However,the current research on improving the quality of projection images using convolutional neural networks is mainly about deblurring and detexturing.There is no datasets and specific solution to remove inter-reflection.To solve the problem,the thesis collects datasets on two immersive projection systems,and uses convolutional neural networks to achieve an end-to-end inter-reflection compensation.The main innovations and work of this thesis are summarized as follows:(1)A new projection compensation network model is proposed to realize an end-to-end inter-reflection compensation,and its feasibility is verified on a single-projection immersive system.This thesis establishes a projection reflection model based on deep learning and uses a convolutional neural network to extract the characteristics of datasets.What's more,the jump connections with deep layer aggregation is innovatively proposed to fuse the different scales of feature maps,so the network can learn complex information of the environmental light and reflection light.(2)A network optimization model based on a super-resolution convolution and a perceptual loss is proposed to improve the compensation ability,and its effectiveness is verified on a multi-projection immersive system.Since the inter-reflection effect in the multi-projectors system is more obvious,a super-resolution convolution and multi-scale loss are added to the projection compensation network model to make the projection images satisfy the human visual perception as much as possible.This mechanism can make the generated compensation image is closer to the target image on the details and improve the projection compensation effect.(3)Two projector-camera immersive display systems are built,and 5 groups of 25,000 datasets containing redundant reflection light are collected in these systems to provide rich datasets for related research.The robustness of the model is tested in the four different types of images.The generalization of the model is tested in four different projection scenes.The experimental results show that the proposed method in the thesis can eliminate the interreflection and achieve high quality reproduction of the projection images.Finally,because the end-user of the immersion projection system is the human vision,besides the objective image quality evaluation,a subjective quality evaluation,MOS(mean opinion score)method,is used.The proposed method in the thesis is superior to other methods in subjective and objective evaluation.