Research Of Image Reconstruction Based On Graph Neural Network

The image reconstruction task of fragments reassembly is a classic problem in the region of computer vision and geometry,its main idea is to reconstruct the images or documents from shredded fragments.The task of restoring the initial information from fragmented visual data is often faced by archaeologist and forensic,such as reconstruction of calligraphy,painting and evidence.However,this problem is tedious and time-consuming for human due to large amounts of pieces.In order to save relevant staffs from the busy reconstruction task,a way to automatically solve the problem is essential.The core problems of fragments reassembly are feature extraction and pairwise matches.In recent years,deep neural network has made much progress in many image classification and recognition tasks.Convolutional neural network and recurrent neural network both play important roles in the field.Comparing with traditional image descriptor,deep learning method shows its strong ability of feature extraction.Therefor we concentrate on the deep neural networks and try to use them to solve the image reconstruction task of fragments reassembly.This paper proposes an algorithm for fragments reassembly problem based on Graph generation strategy.We first transform the question into graph generation problem.The adjacency matrix of generated graph represents the relationship between fragments.Instead of using handcrafted descriptor to extract features from data,we design a preprocessing method and a convolutional neural network to catch the main contents of fragments.This module can learn the distribution of training images and describe the fragments precisely.In order to take good use of the feature,we also provide a corresponding classifier consisted of two recurrent neural networks.It can complete the column elements iteratively according to the current generated graph and image feature.The final output is an adjacency matrix which could indicates the pairwise matches of fragments reassembly.The last step is reassembling the piece images based on the prediction.Our algorithm is verified on the public fragment datasets.We compare the model with several classical descriptors and the latest achievements.The experiments show that our model can learn the ability of finding pairwise matches and make prediction precisely with a low time complexity.And the model is also robust to various special situations.