Multimodal Virtual Reference Frame For Satellite Video Compression

In order to provide high-speed,high-reliability and high-coverage of real-time satellite data services,it is imperative to research an efficient satellite video compression method for satellite video features.The satellite video is mostly geographical background which induced by the repetition of satellite shot of the same area and the background is basically unchanged,which leads to background redundancy.Extracting the background common information of the area,and constructing the background model can effectively remove background redundancy and improve coding efficiency.However,due to environmental factors such as satellite devices and environmental factors such as illumination changes,the background apparent difference is huge.Therefore,to improve the practicality of the background model and the prediction efficiency in coding,this paper defines the background apparent difference as the modal difference,and studies the modal transformation method in the same region to generate the multimodal virtual reference frame.Different from natural videos,satellite videos of the same area have the following characteristics: First,due to the slow change of geomorphology,the background is consistent;Second,due to the large time span of videos and the complex changes of environment,the modal difference is huge.When applying the existing image transformation model to the modal transformation of satellite video,there are the following problems: the structure of the converted picture is inconsistent with the original picture,resulting in low reference efficiency;second,the modal of converted picture has a large difference from the target picture due to the structure and modal of image cannot be stably separated,resulting in a useless background reference frame for compression.In response to the above questions,the main contributions of this paper are as follows:1.For the consistency of structure in the same region and the idea of separating image content and style by multi-style image transfer model,we proposed a modal translation model based on structural consistency constraints.The content structure encoder and modal encoder are used to extract the structural features of the original image and the modal features of the target image respectively.The AdaIN normalization layer and affine transformation are used in the decoder to merge and generate the transformed image in the target modal.When designing the overall optimization function,we add the structural consistency constraint function of the transferred image and the original image.We built the modal data set to test the provided model.The results show that the structures of most of transferred images are consistent,but details are blurred and local color differences in some complex modal transferred images.Compared with the HEVC compression standard,average compression rate is reduced by 8.1825% under the same video quality.2.For the problem of modal and content separation instability,and the understanding of the apparent difference of the same scene image is explored by using the intrinsic image decomposition theory,we propose a modal translation model based on intrinsic image decomposition is proposed.We found that the reflectance and the shading of the intrinsic image decomposition can provide a clearer representation of the modality.Thus,on structure-consistency based modal translation model,an intrinsic image decomposition model [80] is used to obtain the reflectance and the shading of the target image and the transferred image,then add consistency optimization function of the reflectance and the shading.It is verified by experiments that the transferred image of complex modals on the self-built sand disk modal data set is also consistent with the target image.Compared with HEVC under the same video quality,the average compression rate is reduced by 10.39%.