| According to statistics from the World Health Organization in 2017,the number of blind people worldwide is as high as 36 million.It is difficult for blind to distinguish the surrounding scenes and the obstacle in the scene,and the fear of unknown scenes makes them travel rarely.Currently,most guided devices can only detect the distance of obstacles,these devices cannot get information about surrounding scenes.If we integrated the technology of guided scene semantic interpretation into guided system,it could help blind people to travel independently.The technology of guided scene semantic interpretation takes the scene image as input and the scene semantic information as output,and it can obtain the scene category of the image,the category,the orientation and distance of targets.In recent years,with the relative maturity of deep learning,and the image processing technology is improving rapidly,using convolution neural network to extract scene semantic information is the trend of the development of scene semantic interpretation technology.In this paper,we divide the technology of guided scene semantic interpretation into two modules: the scene recognition and the target detection and ranging.The former uses the transfer learning to identify the scene category;the latter uses the technology of target detection to extract the category and orientation information of the targets,and uses the technology of binocular distance measurement to extract the distance information of the targets.The research content of this paper is as follows:(1)For scene recognition issues,we make a data set of guided scene.In order to avoid over-fitting problems during training caused by insufficient data samples,we use the method of transfer learning to train the network model.First,we choose three pre-training models,including VGG19,Inception V3 and Res Net50.We use the data set of guided scene and the method of transfer learning to freeze all convolutional layers and train the top layer of network model.Then explore the effect of number of frozen layers on the recognition results on Res Net50.Finally,the superiority of transfer learning in scene recognition tasks was verified by experiments.(2)For the target detection problem,we take the lightweight convolutional neural network Mobile Net as the fundermental network which is in front of the network,and build a lightweight object detection network MSSD(Mobile Net-Single Shot Multibox Detector).The MSSD uses convolutional neural networks to extract image features,and predicts the target species and the regression of the location through multi-scale feature map prediction.The self-made guided object detection data set is trained on the MSSD network.The trained module is analyzed qualitatively in the real scene.The speed of detection and accuracy of the model can meet the requirements.(3)For the target ranging problem,the calibration and stereo correction of binocular camera are completed firstly,then we analyze three stereo matching algorithms,including BM、SGBM and GC,and select the SGBM to generate the disparity map.We propose a target ranging strategy based on gray value sorting when the targets are occluded from each other.Then we combine the object detection result obtained and the disparity map obtained by binocular matching.We use this strategy to obtain more accurate distance information.(4)We design a system for semantic interpretation of guided scenes.We use the binocular camera to collect the static image or the dynamic video,and the semantic information of the scene is extracted by computer,and the semantic information is fed back to the blind people by outputting voice.We have tested in a variety of scenarios.We derived from the test results that the system has satisfied the set functional requirements basically from the test results. |
