Research Of Key Technology In Visual Interaction For Online Educational Applications

With the rapid growth of the number of children and the rapid progress of socialization in China,the problem of urban children left behind is getting worse.To alleviate this problem,the market was born a variety of intelligent online education applications,such as desktop education robot,homework to help and so on.However,the interactive way of the online application of education on the market mainly for manual contact,and non-contact interactive way such as visual interaction more in line with the user's natural behavior.However,it's hard to apply visual interaction method because of the existence of some factors such as the change of light,the change of angle,the occlusion and so on.So,it is very worthwhile to research how to apply the visual interaction method to the online education application.In this paper,with the in-depth study of computer vision related algorithms,we work in two aspects of image locating and segmenting and image retrieval,aim at the difficulties of desktop education robot in visual interaction,and it can be specifically summarized as follows:Firstly,an algorithm of image locating and segmenting is proposed to solve the interference factors such as background,illumination,noise and occlusion in the target image.This algorithm uses Graph cut algorithm to create the image structure mask matrix,and uses this matrix for edge detection and hough transform line fitting and then rotation correction and occlusion contour removal,thus dividing the target image.Based on this algorithm,this paper uses different mobile phones and cameras in different scenes to shoot and extract the target book images,and build an image data set with 1200 images.Secondly,in order to reduce the semantic gap between low-level visual features and human high-level cognition,this paper proposes a 12-layer residual network DreNet,and the Batch Normalization layer is added to accelerate the convergence rate of the model.The network extracts and combines the image content characteristics by multi-layer convolution to learn the potential semantic features of the book image and this model can achieve 98.2% classification accuracy on the book image data set.Thirdly,in view of the shortcomings that traditional features can not rich express the content of the image,this paper learns from the idea of transfer learning and proposed a depth feature extraction factor by improving the DreNet model and introducing PCA dimensionality reduction.Image retrieval experiments show that the feature extraction factor has a great improvement over the traditional features in accuracy,recall rate and F1 value in different lengths of return list.This paper proposed a solution for different difficulties of the current visual interaction,with the in-depth study of image segmentation and image retrieval.In addition,this paper build an intelligent education interactive system,based on python tkinter library,image segmentation algorithm,depth feature extraction factor and speech recognition API from baidu,and achieve the function of image reading,image locating and segmenting,image retrieval,speech recognition and so on.