| Since sketch images usually lack feature information such as color and texture,recognition models designed for natural images are usually weak in processing sketch images,which also leads to the poor performance of modern computer vision systems for complex multimodal tasks related to sketches.Currently,the commonly used sketch recognition methods can be divided into traditional methods based on manual feature extraction and deep learning-based methods.Most of the traditional methods need to extract features manually and encode them according to the feature types.These methods require complex pre-processing processes and are difficult to use for solving complex problems in the real world,and the manually extracted shallow features cannot fully express the content of sketches and are difficult to produce desirable results.Deep learning-based methods employ a neural network structure to extract and learn the salient features of sketches.In recent years,these deep learning-based sketch recognition methods have achieved good recognition results,but there are still problems such as large number of model parameters and high computational complexity.Moreover,it is difficult for these models to ensure the full utilization of sketch features in the learning process.In this paper,to address the above problems,a lightweight sketch recognition method based on deep learning is investigated,and the main works are as follows:(1)An efficient sketch feature extraction module is built to extract rich sketch features by less computational effort.For the problem of sketches lacking color and texture information and sparse visual cues,a larger first convolution kernel is used to extract more information from the original sketches,and then a multi-level neural network structure is used to extract and combine the low-level features and high-level features of the sketches.(2)A sketch recognition method is proposed based on dual attention mechanism.For the uneven distribution of strokes in sketches,the channel attention module is used to determine the important feature information of sketches in the channel dimension and enhance the information-rich channels,and then the spatial attention module is employed to find the spatial locations where these useful features are distributed and assign more reasonable weights to these locations.The model is guided to focus on the more valuable regions in space and ignore the insignificant factors such as background and weak discriminant factors,the results of subjective and objective evaluations verify the effectiveness of the proposed method.(3)An effective and lightweight convolutional neural network for sketch recognition(Elight-SRNet)is presented,using an improved lightweight feature extraction module with a compression operation to further reduce the number of input channels and the computational complexity.Experiments on TU-Berlin,Sketchy and Quick Draw Extended datasets show that the Elight-SRNet can obtain competitive recognition accuracy and is suitable for mobile terminals or embedded systems with limited computational power and memory capacity. |
