Image Set Compression Based On 3D Model

With the wide adoption of cloud storage and computing, explosively growing im-ages on the web raise big challenge for service providers trying to store and manage them efficiently. Meanwhile, those images often include large amount of similar ones, which have been organized and stored as image sets. Among those similar images, there are many repeats of scenes and objects. For those in the same set, there are quite similarities while some differences as well. The state-of-the-art image set compression methods adopt 2D deformation model and are not accurate enough to efficiently com-press general image set with larger difference. And they often use a video-like coding order, which produces high coding complexity. Therefore, to exploit the redundancy among the similar images in a set, we propose a novel image set compression scheme based on 3D model.Following Wyner’s theory, it is proposed to first represent the "common informa-tion" of multiple images, and then encode each image separately based on the common information. To be more specific, the common information is represented by a sparse 3D point cloud which is extracted from multiple images that photographed the same scene via Photo Tourism system and thus depicts the geometry of that scene. In the mean time, several canonical images are selected from the image set to provide the tex-ture of that scene. Given the above representations of common information, each image is processed separately by first selecting a reference image from the canonical images, then warping the reference as guided by the 3D point cloud, and last encoding the image with a warped reference as its prediction using HEVC inter coding.The proposed approach is implemented and tested on 3 typical image sets:Rock-Boat, Herzjesu and Notre Dame. Experiments results show the superior performance of our proposed compression scheme in comparison with JPEG, HEVC intra and exist-ing image set compression scheme. On personal photo album RockBoat and Herzjesu, our scheme achieves about 10% bit-rate reduction compared with HEVC intra, and is almost 3 times better than JPEG. Considering the 3D model’s bits, our scheme is still 0.3% better than the state-of-the-art image set compression scheme. On web image set Notre Dame with more differences, our scheme achieves about 1.8% bit-rate reduction compared with HEVC intra, while existing methods are unable to compress them of multiple resolutions. Besides, each module’s efficiency is tested. The canonical image selection contributes significant bit-rate reduction. For smaller set like RockBoat and Herzjesu,3-5 canonical images are usually sufficient to cover all angles; for larger set like Notre Dame, experimental results show that choosing 5%-10% of the total images as the canonical ones could lead to almost optimal compression efficiency. The exper-iments on Notre Dame also show the proposed scheme significantly outperforms the JPEG and HEVC intra as long as high quality reference is provided. The average bit-rate reduction is 15.1% while the reduction for one image can reach as high as 75.1% in comparison with HEVC intra on the tested 30 images of Notre Dame. Meanwhile, the perceptual quality of the reconstructed images is improved.To sum up, the proposed image set compression scheme based on 3D model is able to generate more accurate predictions and significantly improve the compression efficiency. What’s more, the scheme could provide almost "Random Access" in decod-ing process because only one canonical image is selected as the reference for encoding one image, which greatly reduces the complexity and is suitable for many real-world applications such as image social network, virtual museum tour and so on.