Temporal And Spatial Neural Network For VideoSAR Semantic Segmentation

As a typical active imaging radar,synthetic aperture radar(SAR)can obtain high-resolution images of detected area by 2D inversion.With the development of SAR imaging technology,not only can a single detected area image be used for analysis,but also an image sequence of continuous detected area,that is,video synthetic aperture radar(Video SAR),which provides richer information for subsequent processing.In recent years,as an emerging algorithm,deep neural networks have been used in many fields because of its powerful learning capabilities,and neural networks have solved many problems that predecessors could not solve.With the deep research of deep neural networks,we have made great breakthroughs in the SAR image semantic segmentation task.However,unlike traditional SAR images,Video SAR has unique timing features.So,Video SAR semantic segmentation remains a very challenging task.In this paper,on the basis of combining traditional SAR image features and convolutional neural networks,we come up with some new strategies for Video SAR semantic segmentation by using timing features.First of all,the convolution kernel of backbone network is optimized to a certain degree,and the feature extraction of Video SAR is achieved by stacking 3D dilated residual network block.3D convolutional neural networks can effectively utilize the temporal and spatial features in Video SAR.Short cut structure can improve the learning ability of neural network.At the same time,the dilated convolution kernel can increase the receptive field of convolution kernels,which is very beneficial for semantic segmentation tasks.Secondly,in order to more fully integrate the temporal and spatial features of Video SAR,we apply Long Short-Term Memory(LSTM)network that are very effective at extracting timing features.However,traditional LSTM network can only process one dimensional feature vectors and cannot be transplanted into semantic segmentation models.In our model,we have optimized the structure of LSTM network to make the semantic segmentation model achieve good results.Finally,when processing Video SAR data set,we combined the optical flow features of the Video SAR data,established an optical flow feature model,and implanted it into the semantic segmentation model.Experiments show that the optical flow features are indispensable when processing Video SAR data,a deep neural network model which incorporates optical flow features has made improvements for Video SAR semantic segmentation tasks.We used the Video SAR data published by Sandia National Laboratories,annotated the video content,and amplified the data.A Video SAR data set was established and many experiments were performed on this data set.The experiment results fully demonstrate the effectiveness of the method of this paper in Video SAR semantic segmentation task.